Primary-subordinate camera focus based on lens position sensing

ABSTRACT

Various embodiments disclosed herein include techniques for maintaining multiple cameras in focus on same objects and/or at same distances. In some examples, a subordinate camera may be configured to focus based on the focus of a primary camera. For instance, a focus relationship between the primary camera and the subordinate camera may be determined. The focus relationship may characterize the trajectory of the lens position of the subordinate camera with respect to the lens position of the primary camera. In various examples, the focus relationship may be updated.

This application is a continuation of U.S. application Ser. No.17/091,825, filed Nov. 6, 2020, which is a continuation of U.S.application Ser. No. 16/586,781, filed Sep. 27, 2019, now U.S. Pat. No.10,830,990, which is a continuation of U.S. application Ser. No.15/710,747, filed Sep. 20, 2017, now U.S. Pat. No. 10,429,608, whichclaims benefit of priority to U.S. Provisional Application No.62/398,910, filed Sep. 23, 2016, which are hereby incorporated byreference in their entirety.

BACKGROUND Technical Field

This disclosure relates generally to focusing multiple cameras and morespecifically to focusing multiple cameras on a same image subject basedat least in part on a focus relationship between the cameras.

Description of the Related Art

The advent of small, mobile multipurpose devices such as smartphones andtablet or pad devices has resulted in a need for high-resolution, smallform factor cameras for integration in the devices. Some small formfactor cameras may incorporate optical image stabilization (OIS)mechanisms that may sense and react to external excitation/disturbanceby adjusting location of the optical lens on the x and/or y axis in anattempt to compensate for unwanted motion of the lens. Some small formfactor cameras may incorporate an autofocus (AF) mechanism whereby theobject focal distance can be adjusted to focus an object plane in frontof the camera at an image plane to be captured by the image sensor. Insome such autofocus mechanisms, the optical lens is moved as a singlerigid body along the optical axis (or the z axis) of the camera torefocus the camera.

In addition, high image quality is easier to achieve in small formfactor cameras if lens motion along the optical axis is accompanied byminimal parasitic motion in the other degrees of freedom, for example onthe X and Y axes orthogonal to the optical (Z) axis of the camera. Thus,some small form factor cameras that include autofocus mechanisms mayalso incorporate optical image stabilization (OIS) mechanisms that maysense and react to external excitation/disturbance by adjusting locationof the optical lens on the X and/or Y axis in an attempt to compensatefor unwanted motion of the lens. In such systems, knowledge of theposition of the lens is useful.

SUMMARY OF EMBODIMENTS

Various implementations disclosed herein include techniques formaintaining multiple cameras (e.g., dissimilar cameras) in focus on sameobjects and/or at same distances. In some examples, a subordinate cameramay be configured to focus based on the focus of a primary camera. Forinstance, a focus relationship between the primary camera and thesubordinate camera may be determined. The focus relationship maycharacterize the trajectory of the lens position of the subordinatecamera with respect to the lens position of the primary camera. Invarious examples, the focus relationship may be updated from time totime.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a perspective view of an example camera system thatincludes an example primary camera unit and an example subordinatecamera unit, in accordance with some embodiments. The examplesubordinate camera unit of FIG. 1 may be focused on an image subjectbased on a focus relationship between the example subordinate cameraunit and the example primary camera unit, in accordance with someembodiments.

FIG. 2 is a flowchart of an example method of focusing a subordinatecamera based on a focus relationship between the subordinate camera anda primary camera, in accordance with some embodiments.

FIG. 3 is a flowchart of an example method of determining a focusrelationship between a primary camera and a subordinate camera, inaccordance with some embodiments.

FIG. 4 is a flowchart of an example method of updating a focusrelationship between a primary camera and a subordinate camera, inaccordance with some embodiments.

FIG. 5 is a flowchart of an example method of determining an updatedfocus relationship between a primary camera and a subordinate camera, inaccordance with some embodiments. The example method of FIG. 5 , thesubordinate camera may be focused based on a constrained focus range, inaccordance with some embodiments.

FIG. 6 is a flowchart of an example method of updating an offset term ofa focus relationship, in accordance with some embodiments.

FIG. 7 is a flowchart of an example method of maintaining focuscontinuity across a transition from one camera mode to another cameramode, in accordance with some embodiments.

FIGS. 8A-8B illustrate, via graphs, example characteristics of a focusrelationship between positioning of a primary camera and positioning ofa subordinate camera lens, in accordance with some embodiments.

FIG. 9 is a flow block diagram of an example method of focusing asubordinate camera using an adaptive lens model that is based on a focusrelationship between the subordinate camera and a primary camera, inaccordance with some embodiments.

FIG. 10 is a block diagram of an example method of calculating atemperature-corrected position of a primary camera lens and/or asubordinate camera lens, in accordance with some embodiments.

FIG. 11 is a block diagram of an example estimator for estimating afocus relationship between a primary camera and a subordinate camera, inaccordance with some embodiments.

FIG. 12 illustrates a schematic side view of an example camera modulehaving an example voice coil motor (VCM) actuator for moving an opticalpackage, in accordance with some embodiments.

FIG. 13 illustrates a block diagram of an example portable multifunctiondevice that may include a primary camera and a subordinate camera, inaccordance with some embodiments.

FIG. 14 illustrates an example portable multifunction device that mayinclude a primary camera and a subordinate camera, in accordance withsome embodiments.

FIG. 15 illustrates an example computer system that may include aprimary camera and a subordinate camera, according to some embodiments.

This specification includes references to “one embodiment” or “anembodiment.” The appearances of the phrases “in one embodiment” or “inan embodiment” do not necessarily refer to the same embodiment.Particular features, structures, or characteristics may be combined inany suitable manner consistent with this disclosure.

“Comprising.” This term is open-ended. As used in the appended claims,this term does not foreclose additional structure or steps. Consider aclaim that recites: “An apparatus comprising one or more processorunits. . . . ” Such a claim does not foreclose the apparatus fromincluding additional components (e.g., a network interface unit,graphics circuitry, etc.).

“Configured To.” Various units, circuits, or other components may bedescribed or claimed as “configured to” perform a task or tasks. In suchcontexts, “configured to” is used to connote structure by indicatingthat the units/circuits/components include structure (e.g., circuitry)that performs those task or tasks during operation. As such, theunit/circuit/component can be said to be configured to perform the taskeven when the specified unit/circuit/component is not currentlyoperational (e.g., is not on). The units/circuits/components used withthe “configured to” language include hardware—for example, circuits,memory storing program instructions executable to implement theoperation, etc. Reciting that a unit/circuit/component is “configuredto” perform one or more tasks is expressly intended not to invoke 35U.S.C. § 112, sixth paragraph, for that unit/circuit/component.Additionally, “configured to” can include generic structure (e.g.,generic circuitry) that is manipulated by software and/or firmware(e.g., an FPGA or a general-purpose processor executing software) tooperate in manner that is capable of performing the task(s) at issue.“Configure to” may also include adapting a manufacturing process (e.g.,a semiconductor fabrication facility) to fabricate devices (e.g.,integrated circuits) that are adapted to implement or perform one ormore tasks.

“First,” “Second,” etc. As used herein, these terms are used as labelsfor nouns that they precede, and do not imply any type of ordering(e.g., spatial, temporal, logical, etc.). For example, a buffer circuitmay be described herein as performing write operations for “first” and“second” values. The terms “first” and “second” do not necessarily implythat the first value must be written before the second value.

“Based On.” As used herein, this term is used to describe one or morefactors that affect a determination. This term does not forecloseadditional factors that may affect a determination. That is, adetermination may be solely based on those factors or based, at least inpart, on those factors. Consider the phrase “determine A based on B.”While in this case, B is a factor that affects the determination of A,such a phrase does not foreclose the determination of A from also beingbased on C. In other instances, A may be determined based solely on B.

DETAILED DESCRIPTION

Various implementations disclosed herein include techniques formaintaining multiple cameras (e.g., dissimilar cameras) in focus on sameobjects and/or at same distances. In some examples, a subordinate cameramay be configured to focus based on the focus of a primary camera. Forinstance, a focus relationship between the primary camera and thesubordinate camera may be determined. The focus relationship maycharacterize the trajectory of the lens position of the subordinatecamera with respect to the lens position of the primary camera. Invarious examples, the focus relationship may be updated from time totime.

Some embodiments include a camera system. The camera system may includea primary camera and a subordinate camera. The primary camera mayinclude a first set of one or more lenses (also referred to herein as a“primary camera lens”) that define a first optical axis and a firstfocal length. In some examples, the primary camera may include a firstactuator (e.g., a voice coil motor (VCM) actuator) configured to movethe primary camera lens along the first optical axis to enable focusingfor the primary camera. The subordinate camera may include a second setof one or more lenses (also referred to herein as a “subordinate cameralens”) that define a second optical axis and a second focal length. Invarious cases, the second optical axis of the subordinate camera may beparallel, or substantially parallel, to the first optical axis of theprimary camera. Additionally, or alternatively, the second focal lengthof the subordinate camera may be different than the first focal lengthof the primary camera. In some examples, the subordinate camera mayinclude a second actuator (e.g., a VCM actuator) configured to move thesubordinate camera lens along the second optical axis to enable focusingfor the subordinate camera. Although the primary camera and thesubordinate camera are described herein as possibly having differentfocal lengths, the primary camera and the subordinate camera mayadditionally or alternatively be similar, identical, and/or dissimilarin other ways (e.g., by having different minimum focus distances).

In some embodiments, the camera system may include one or moreprocessors and memory. The memory may include program instructions that,when executed by the one or more processors, cause the one or moreprocessors to perform operations. In some implementations, theoperations may include determining a focus relationship thatcharacterizes focus positioning of the subordinate camera lens withrespect to focus positioning of the primary camera lens. In someexamples, the operations may include causing the primary camera toindependently focus on an image subject. For instance, the primarycamera may be focused on the image subject based at least in part onimage content corresponding to the image subject. Furthermore, theoperations may include causing the subordinate camera to focus on theimage subject based at least in part on the focus relationship.

In some examples, determination of the focus relationship may includecausing the primary camera and the subordinate camera to focus on imagesubjects (e.g., same image subjects at same distances). A first set ofposition data and a second set of position data may be obtained based onfocusing the primary camera and focusing the subordinate camera,respectively. The first set of position data may correspond torespective focus positions of the primary camera lens based at least inpart on focusing the primary camera on the image subjects. The secondset of position data may correspond to respective focus positions of thesubordinate camera lens based at least in part on focusing thesubordinate camera on the image subjects. In various implementations,the focus relationship may be determined based at least in part on thefirst set of position data and the second set of position data.

According to various embodiments, the focus relationship may include anoffset term that is variable based at least in part on one or moreparameters corresponding to the primary camera and/or the subordinatecamera. For instance, the parameters may include a respectivetemperature associated with the primary camera and/or the subordinatecamera. In some cases, the parameters may include a first temperatureassociated with the primary camera lens and/or a second temperatureassociated with the subordinate camera lens. As lens temperatures maychange during operation of the cameras, the offset term of the focusrelationship may also change.

In some embodiments, the focus relationship between the primary cameraand the subordinate camera may be updated. In some cases, the focusrelationship may be updated to account for a change in the offset termof the focus relationship, which, in turn, may be caused by a change inone or more parameters associated with the primary camera and/or thesubordinate camera. In some examples, the focus relationship may bedetermined during a first time period, and then may be updated during asecond time period after the first time period. In some instances, theterms “first time period” and “second time period” are used herein asexample time periods corresponding to determining the focus relationshipand updating the focus relationship, respectively, but the focusrelationship may also be updated during other time periods in arecursive process.

The operations for updating the focus relationship may include causing,during the second time period, the primary camera to focus on at leastone subject image. A third set of position data may be obtained duringthe second time period. The third set of position data may correspond toone or more focus positions of the primary camera lens based at least inpart on causing the primary camera to focus on the image subject(s).Furthermore, the operations may include causing, during the second timeperiod, the subordinate camera to focus on the image subject(s) (e.g.,the same image subject(s) as those focused on by the primary cameraduring the second time period). A fourth set of position data may beobtained during the second time period. The fourth set of position datamay correspond to one or more focus positions of the subordinate cameralens based at least in part on focusing the subordinate camera on theimage subject(s). In various implementations, the second focusrelationship may be determined based at least in part on the third setof position data and the fourth set of position data.

Updating the focus relationship may include updating the offset term ofthe focus relationship. In some examples, a current state of one or moreparameters (e.g., temperature) associated with the primary camera and/orthe subordinate camera may be determined, and the offset term of thefocus relationship may be updated based at least in part on the currentstate of the parameter(s).

In some embodiments, the camera system may include a primary camera andmultiple subordinate cameras. For instance, the subordinate cameradescribed above may be a first subordinate camera, and the camera systemmay further include a second subordinate camera. The second subordinatecamera may include a third set of one or more lenses (also referred toherein as a “second subordinate camera lens”) that define a thirdoptical axis and a third focal length. In various cases, the thirdoptical axis of the second subordinate camera may be parallel, orsubstantially parallel, to the first optical axis of the primary cameraand/or to the second optical axis of the first subordinate camera.Additionally, or alternatively, the third focal length of the secondsubordinate camera may be different than the first focal length of theprimary camera and/or the second focal length of the first subordinatecamera. In some examples, the second subordinate camera may include athird actuator (e.g., a VCM actuator) configured to move the secondsubordinate camera lens along the second optical axis to enable focusingfor the second subordinate camera.

In some examples, a focus relationship (also referred to herein as a“second subordinate camera focus relationship”) between the secondsubordinate camera and one or more of the primary camera or the firstsubordinate camera may be determined. For instance, a second subordinatecamera focus relationship may characterize focus positioning of thesecond subordinate camera lens with respect to focus positioning of theprimary camera lens. Additionally, or alternatively, a secondsubordinate camera focus relationship may characterize focus positioningof the second subordinate camera lens with respect to focus positioningof the first subordinate camera lens. The operations may include causingthe second subordinate camera to focus on an image subject based atleast in part on one or more second subordinate camera focusrelationships.

Some embodiments include a method. The method may include focusing aprimary camera on an image subject based at least in part on imagecontent corresponding to the image subject. The primary camera mayinclude a primary camera lens that defines a first optical axis and afirst focal length. A focus position of the primary camera lens, atwhich the primary camera is focused on the image subject, may bedetermined. Furthermore, the method may include focusing a subordinatecamera on the image subject based at least in part on the focus positionof the primary camera lens and a focus relationship between thesubordinate camera and the primary camera. The subordinate camera mayinclude a subordinate camera lens that defines a second optical axis anda second focal length. In some examples, the second focal length of thesubordinate camera may be different than the first focal length of theprimary camera. The focus relationship may characterize focuspositioning of the subordinate camera lens with respect to focuspositioning of the primary camera lens. In this manner, focusing thesubordinate camera may include driving the subordinate camera lensposition without independently focusing the subordinate camera based onthe image content corresponding to the image subject.

In some examples, focusing the primary camera may include moving theprimary camera lens along the first optical axis to a first focusposition at which the first camera is focused on the image subject. Forinstance, the primary camera lens may be moved via a first voice coilmotor (VCM). Furthermore, focusing the subordinate camera may includemoving the subordinate camera lens along the second optical axis to asecond focus position at which the subordinate camera is focused on theimage subject (e.g., based at least in part on the focus relationshipbetween the subordinate camera and the primary camera).

In some examples, determination of the focus relationship may includefocusing the primary camera and the subordinate camera on image subjects(e.g., same image subjects at same distances). A first set of positiondata and a second set of position data may be obtained based at least inpart on focusing the primary camera and focusing the subordinate camera,respectively. The first set of position data may correspond torespective focus positions of the primary camera lens based at least inpart on focusing the primary camera on the image subjects. The secondset of position data may correspond to respective focus positions of thesubordinate camera lens based at least in part on focusing thesubordinate camera on the image subjects. In some cases, the first setof position data may be obtained via one or more position sensors of theprimary camera. Likewise, the second set of position data may beobtained via one or more position sensors of the subordinate camera. Invarious implementations, the focus relationship may be determined basedat least in part on the first set of position data and the second set ofposition data.

According to various embodiments, the focus relationship may include anoffset term that is variable based at least in part on one or moreparameters corresponding to the primary camera and/or the subordinatecamera. For instance, the parameters correspond to parameters that mayaffect focus positioning of the primary camera lens and/or thesubordinate camera lens. For example, the parameters may include a firsttemperature associated with the primary camera lens and/or a secondtemperature associated with the subordinate camera lens.

In some embodiments, the focus relationship between the primary cameraand the subordinate camera may be updated. In some cases, the focusrelationship may be updated to account for a change in the offset termof the focus relationship, which, in turn, may be caused by a change inone or more parameters associated with the primary camera and/or thesubordinate camera. In some examples, the focus relationship may bedetermined during a first time period and may be updated during a secondtime period after the first time period. The primary camera may befocused, during the second time period, on at least one image subject. Athird set of position data may be obtained during the second timeperiod. The third set of position data may correspond to one or morefocus positions of the primary camera lens based at least in part oncausing the primary camera to focus on the image subject(s).Furthermore, the subordinate camera may be focused, during the secondtime period, on the image subject(s) (e.g., the same image subject(s) asthose focused on by the primary camera during the second time period). Afourth set of position data may be obtained during the second timeperiod. The fourth set of position data may correspond to one or morefocus positions of the subordinate camera lens based at least in part onfocusing the subordinate camera on the image subject(s). In variousimplementations, an updated focus relationship may be determined basedat least in part on the third set of position data and the fourth set ofposition data.

Updating the focus relationship may include updating the offset term ofthe focus relationship. In some examples, a current state of one or moreparameters (e.g., temperature) associated with the primary camera and/orthe subordinate camera may be determined, and the offset term of thefocus relationship may be updated based at least in part on the currentstate of the parameter(s).

In some examples, the subordinate camera may be configured to focus bymoving the subordinate camera lens in search of a focus position withina first focus range. While updating the focus relationship (e.g., duringthe second time period), the subordinate camera may be focused on theimage subject(s) by constraining the search, of the focus position(s) ofthe subordinate camera lens at which the subordinate camera is focusedon the image subject(s), to a second focus range. For instance, thesecond focus range may be smaller than the first focus range.Accordingly, by constraining the search to the second focus range, thesubordinate camera may find the focus position faster than it would haveunder an unconstrained search of the larger first focus range. In someinstances, the method may include calculating a confidence level of thefocus relationship, and determining the second focus range (e.g., forupdating the focus relationship) based at least in part on theconfidence level of the focus relationship.

Some embodiments include a mobile device (e.g., a mobile multifunctiondevice). The mobile device may include a primary camera unit and asubordinate camera unit. The primary camera unit may include a firstoptical package that includes one or more lens elements (also referredto herein as a “primary camera unit lens”) that define a first opticalaxis and a first focal length. In some examples, the primary camera unitmay include a first actuator (e.g., a voice coil motor (VCM) actuator)configured to move the primary camera unit lens along the first opticalaxis to enable autofocus functionality for the primary camera unit. Theprimary camera unit may be configured to provide the autofocusfunctionality by independently focusing the primary camera unit onrespective image subjects based at least in part on respective imagecontent corresponding to the respective image subjects.

The subordinate camera unit may include a second optical package thatincludes one or more lenses (also referred to herein as a “subordinatecamera unit lens”) that define a second optical axis and a second focallength. In various cases, the second optical axis of the subordinatecamera unit may be parallel, or substantially parallel, to the firstoptical axis of the primary camera unit. Additionally, or alternatively,the second focal length of the subordinate camera unit may be differentthan the first focal length of the primary camera unit. For instance, insome embodiments, one of the primary camera unit or the subordinatecamera unit may be a telephoto lens camera, and the other of the primarycamera unit or the subordinate camera unit may be a wide angle lenscamera. In some examples, the subordinate camera unit may include asecond actuator (e.g., a VCM actuator) configured to move thesubordinate camera unit lens along the second optical axis to enableautofocus functionality for the subordinate camera unit. The subordinatecamera unit may be configured to provide the autofocus functionality bymoving, via the second actuator, the subordinate camera unit lens to afocus position based at least in part on a focus relationship, betweenthe subordinate camera unit and the primary camera unit, such that thesubordinate camera unit and the primary camera unit are focused on asame image subject.

The focus relationship may characterize positioning of the subordinatecamera unit lens with respect to positioning of the primary camera unitlens. In some examples, the focus relationship may be determined basedat least in part on the first focal length of the primary camera unitand the second focal length of the subordinate camera unit. Furthermore,the focus relationship may include an offset term that is variable basedat least in part on parameters corresponding to the primary camera unitand/or the subordinate camera unit. In various examples, the parametersmay correspond to parameters that may impact focus positioning of theprimary camera unit lens and/or the subordinate camera unit lens. Forexample, the parameters may include a first temperature associated withthe primary camera unit lens and/or a second temperature associated withthe subordinate camera unit lens.

In some examples, the mobile device may include one or more processorsand memory. The memory may include program instructions that, whenexecuted by the one or more processors, cause the one or more processorsto perform operations. For instance, the operations may includetransitioning from a primary camera mode to a subordinate camera mode.Additionally, or alternatively, the operations may include transitioningfrom the subordinate camera mode to the primary camera mode. In theprimary camera mode, the primary camera unit may be designated as anactive camera for image capturing. Furthermore, in the subordinatecamera mode, the subordinate camera unit may be designated as the activecamera for image capturing. In various embodiments, continuity of focuson the same subject image by the primary camera unit and the subordinatecamera unit may be maintained across the transition from the primarycamera mode to the subordinate camera mode. Similarly, continuity offocus on the same subject image by the primary camera unit and thesubordinate camera unit may be maintained across the transition from thesubordinate camera mode to the primary camera mode.

In various embodiments, the operations may include determining the focusrelationship. Determination of the focus relationship may includecausing the primary camera unit and the subordinate camera unit to focuson image subjects (e.g., same image subjects at same distances). A firstset of position data and a second set of position data may be obtainedbased on focusing the primary camera unit and focusing the subordinatecamera unit, respectively. The first set of position data may correspondto respective focus positions of the primary camera unit lens based atleast in part on focusing the primary camera unit on the image subjects.The second set of position data may correspond to respective focuspositions of the subordinate camera unit lens based at least in part onfocusing the subordinate camera unit on the image subjects. In variousimplementations, the focus relationship may be determined based at leastin part on the first set of position data and the second set of positiondata.

In some embodiments, the operations may include causing the primarycamera unit to capture a first image of an image subject. Furthermore,the operations may include causing the subordinate camera unit tocapture a second image of the image subject. In some cases, a thirdimage may be generated based at least in part on the first image and thesecond image. In various examples, continuity of focus on the subjectimage by the primary camera unit and the subordinate camera unit ismaintained across the operations of causing the primary camera unit tocapture the first image and causing the subordinate camera unit tocapture the second image.

In various embodiments, the operations may include periodically updatingthe focus relationship. For instance, updating the focus relationshipmay include updating the offset term of the focus relationship based atleast in part on a change in value of one or multiple parameterscorresponding to the primary camera unit and/or the second camera unit.

Reference will now be made in detail to embodiments, examples of whichare illustrated in the accompanying drawings. In the following detaileddescription, numerous specific details are set forth in order to providea thorough understanding of the present disclosure. However, it will beapparent to one of ordinary skill in the art that some embodiments maybe practiced without these specific details. In other instances,well-known methods, procedures, components, circuits, and networks havenot been described in detail so as not to unnecessarily obscure aspectsof the embodiments.

It will also be understood that, although the terms first, second, etc.may be used herein to describe various elements, these elements shouldnot be limited by these terms. These terms are only used to distinguishone element from another. For example, a first contact could be termed asecond contact, and, similarly, a second contact could be termed a firstcontact, without departing from the intended scope. The first contactand the second contact are both contacts, but they are not the samecontact.

The terminology used in the description herein is for the purpose ofdescribing particular embodiments only and is not intended to belimiting. As used in the description and the appended claims, thesingular forms “a”, “an” and “the” are intended to include the pluralforms as well, unless the context clearly indicates otherwise. It willalso be understood that the term “and/or” as used herein refers to andencompasses any and all possible combinations of one or more of theassociated listed items. It will be further understood that the terms“includes,” “including,” “comprises,” and/or “comprising,” when used inthis specification, specify the presence of stated features, integers,steps, operations, elements, and/or components, but do not preclude thepresence or addition of one or more other features, integers, steps,operations, elements, components, and/or groups thereof.

As used herein, the term “if” may be construed to mean “when” or “upon”or “in response to determining” or “in response to detecting,” dependingon the context. Similarly, the phrase “if it is determined” or “if [astated condition or event] is detected” may be construed to mean “upondetermining” or “in response to determining” or “upon detecting [thestated condition or event]” or “in response to detecting [the statedcondition or event],” depending on the context.

FIG. 1 illustrates a perspective view of an example camera system 100that includes an example primary camera unit 102 (also referred toherein as a “primary camera” or a “primary camera module”) and anexample subordinate camera unit 104 (also referred to herein as a“subordinate camera” or a “subordinate camera module”), in accordancewith some embodiments.

In various examples, the primary camera unit 102 may include a firstoptical package that includes a first set of one or more lenses 106(also referred to herein as a “primary camera lens”) that define a firstoptical axis and a first focal length. In some cases, the first focallength may be adjustable within a range of focal lengths. In someexamples, the primary camera unit 102 may include a first actuator(e.g., a voice coil motor (VCM) actuator as illustrated in FIG. 12 )configured to move the primary camera lens 106 along the first opticalaxis to enable focusing for the primary camera unit 102. For instance,the primary camera lens 106 may move along the first optical axis asindicated by the primary camera lens movement arrow 108.

The subordinate camera unit 104 may include a second set of one or morelenses 110 (also referred to herein as a “subordinate camera lens”) thatdefine a second optical axis and a second focal length. In variouscases, the second optical axis of the subordinate camera unit 104 may beparallel, or substantially parallel, to the first optical axis of theprimary camera unit 102. Additionally, or alternatively, the secondfocal length of the subordinate camera unit 104 may be different thanthe first focal length of the primary camera unit 102. For instance, insome embodiments, one of the primary camera unit 102 or the subordinatecamera unit 104 may be a telephoto lens camera, and the other of theprimary camera unit 102 or the subordinate camera unit 104 may be a wideangle lens camera. In some cases, the second focal length may beadjustable within a range of focal lengths. In some examples, thesubordinate camera unit 104 may include a second actuator (e.g., a VCMactuator as illustrated in FIG. 12 ) configured to move the subordinatecamera lens 110 along the second optical axis to enable focusing for thesubordinate camera unit 104. For instance, the subordinate camera lens110 may move along the second optical axis as indicated by thesubordinate camera lens movement arrow 112. Although the primary cameraunit 102 and the subordinate camera unit 104 are described herein aspossibly having different focal lengths, the primary camera unit 102 andthe subordinate camera unit 104 may additionally or alternatively besimilar, identical, and/or dissimilar in other ways (e.g., by havingdifferent minimum focus distances).

In some embodiments, the camera system 100 may include one or moreprocessors and memory (e.g., as described below with reference to FIGS.13 and 15 ). The memory may include program instructions that, whenexecuted by the one or more processors, cause the one or more processorsto perform operations, e.g., one or more of the operations describedbelow with reference to FIGS. 2-6 and 9-11 . Additionally, oralternatively, the camera system 100 may include a controller (notshown) that performs the operations.

In some implementations, the operations may include causing the primarycamera unit 102 and the subordinate camera unit 104 to focus on objects.For instance, the primary camera unit 102 may be focused on an imagesubject 114 (e.g., the sailboat depicted in FIG. 1 ) within a scene 116(e.g., the coast-water-sailboat-sky scene depicted in FIG. 1 ). In someexamples, the primary camera unit 102 may be focused on the imagesubject 114 based at least in part on image content corresponding to theimage subject 114. For instance, focusing of the primary camera unit 102on the image subject 114 may include adjusting the position of theprimary camera lens 108 based at least in part on one or more metrics(e.g., sharpness, contrast, etc.). In a non-limiting example, theposition of the primary camera lens 108 may be adjusted to satisfy athreshold image metric (e.g., a threshold sharpness, a thresholdcontrast, etc.) that indicates that the primary camera unit 102 isfocused on the image subject 114. Additionally or alternatively, theprimary camera unit 102 may be focused on the image subject 114 based atleast in part on one or more autofocus techniques (e.g., phasedetection, contrast detection, laser autofocus, etc.).

In some implementations, the subordinate camera unit 104 may be focusedon the same image subject 114. In some instances, the subordinate cameraunit 104 may be focused on the image subject 114 when the subordinatecamera unit 104 is at or about the same distance away from the imagesubject 114 as the primary camera unit 102. For example, the subordinatecamera unit 104 may be adjacent to the primary camera unit. Furthermore,the subordinate camera unit 104 may be focused on the image subject 114at or about the same time as the primary camera unit is focused on theimage subject 114.

In various implementations, the subordinate camera unit 104 may befocused on the same image subject 114 based at least in part on a lensmodel 118 that takes into account a focus relationship 120 between theprimary camera unit 102 and the subordinate camera unit 104. The focusrelationship 120 may characterize focus positioning of the subordinatecamera lens 110 with respect to focus positioning of the primary cameralens 106. For example, the lens model 118 may receive the primary cameralens focus position 122 as an input, and output, based at least in parton the focus relationship 120, the subordinate camera lens focusposition 124. As such, in various implementations, the subordinatecamera unit 104 may be maintained in focus on the same image subject 114as the primary camera unit 102 without independently searching for afocus position over a focus range of the subordinate camera lens 110.Thus, by using the lens model 118 and techniques described herein, thesubordinate camera unit 104 may be focused on image subjects faster thanin some conventional focusing techniques in which a camera relies onindependently searching for a focus position over a focus range. Asdiscussed in further detail below with reference to FIGS. 4-6 and 9-11 ,the lens model 118 and/or the focus relationship 120 may be updated fromtime to time.

Although FIG. 1 depicts a single primary camera unit 102 and a singlesubordinate camera unit 104, the camera system may, in some embodiments,include multiple primary camera units and/or multiple subordinate cameraunits. For instance, the subordinate camera unit 104 may be a firstsubordinate camera, and the camera system 100 may further include asecond subordinate camera (not shown). The second subordinate camera mayinclude a third set of one or more lenses (also referred to herein as a“second subordinate camera lens”) that define a third optical axis and athird focal length. In various cases, the third optical axis of thesecond subordinate camera may be parallel, or substantially parallel, tothe first optical axis of the primary camera and/or to the secondoptical axis of the first subordinate camera. Additionally, oralternatively, the third focal length of the second subordinate cameramay be different than the first focal length of the primary cameraand/or the second focal length of the first subordinate camera. In someexamples, the second subordinate camera may include a third actuator(e.g., a VCM actuator) configured to move the second subordinate cameralens along the second optical axis to enable focusing for the secondsubordinate camera.

In some examples, a focus relationship (also referred to herein as a“second subordinate camera focus relationship”) between the secondsubordinate camera and one or more of the primary camera or the firstsubordinate camera may be determined. For instance, a second subordinatecamera focus relationship may characterize focus positioning of thesecond subordinate camera lens with respect to focus positioning of theprimary camera lens. Additionally, or alternatively, a secondsubordinate camera focus relationship may characterize focus positioningof the second subordinate camera lens with respect to focus positioningof the first subordinate camera lens. The operations may include causingthe second subordinate camera to focus on an image subject based atleast in part on one or more second subordinate camera focusrelationships.

FIG. 2 is a flowchart of an example method 200 of focusing a subordinatecamera (e.g., the subordinate camera unit 104 described above withreference to FIG. 1 ) based on a focus relationship between thesubordinate camera and a primary camera (e.g., the primary camera unit102 described above with reference to FIG. 1 ), in accordance with someembodiments.

At 202, the method 200 may include focusing the primary camera on animage subject. In some examples, the primary camera may be focused onthe image subject based at least in part on image content correspondingto the image subject. For instance, focusing of the primary camera onthe image subject may include adjusting the position of the primarycamera lens based at least in part on one or more metrics (e.g.,sharpness, contrast, etc.). In a non-limiting example, the position ofthe primary camera lens may be adjusted to satisfy a threshold imagemetric (e.g., a threshold sharpness, a threshold contrast, etc.) thatindicates that the primary camera is focused on the image subject.Additionally or alternatively, the primary camera may be focused on theimage subject based at least in part on one or more autofocus techniques(e.g., phase detection, contrast detection, laser autofocus, etc.). At204, the method 200 may include determining a focus position of theprimary camera lens at which the primary camera is focused on the imagesubject. At 204, the method 200 may include focusing the subordinatecamera on the image subject. For instance, the subordinate camera may befocused on the image subject based at least in part on the focusposition of the primary camera and a focus relationship between thesubordinate camera and the primary camera.

FIG. 3 is a flowchart of an example method 300 of determining a focusrelationship between a primary camera (e.g., the primary camera unit 102described above with reference to FIG. 1 ) and a subordinate camera(e.g., the subordinate camera unit 104 described above with reference toFIG. 1 ), in accordance with some embodiments.

At 302, the method 300 may include focusing the primary camera onsubject images. In various embodiments, the primary camera may befocused independently of the focusing of the subordinate camera, e.g.,using one or more of the techniques described above with reference toFIGS. 1 and 2 . At 304, the method 300 may include obtaining a first setof position data corresponding to respective focus positions of theprimary camera lens. For instance, the first set of position data may beobtained at least partly via one or more position sensors (e.g., theposition sensors 1208 described below with reference to FIG. 12 ) of theprimary camera. In some examples, the position sensors may include oneor more magnetic field sensors (e.g., Hall sensors, tunnelingmagnetoresistance (TMR) sensors, giant magnetoresistance (GMR) sensors,etc.). For instance, the primary camera may include a voice coil motor(VCM) actuator having one or more coils and one or more magnets, e.g.,as described below with reference to FIG. 12 . The coils may beconfigured to receive a current and magnetically interact with one ormore magnets to produce Lorentz forces that cause the primary cameralens to move along an optical axis. One or more position sensor magnets(e.g., the position sensor magnets 1214 described below with referenceto FIG. 12 ) may be coupled to the primary camera lens such that theposition sensor magnets move along with the primary camera lens. One ormore magnetic field sensors may be used to detect the position of theposition sensor magnets, thereby enabling position sensing of theprimary camera lens.

At 306, the method 300 may include focusing the subordinate camera onthe subject images. In various embodiments, the subordinate camera maybe focused independently of the focusing of the primary camera. In someexamples, the primary camera may be focused on the image subject basedat least in part on image content corresponding to the image subject.For instance, focusing of the subordinate camera on the image subjectsmay include adjusting the position of the primary camera lens based atleast in part on one or more metrics (e.g., sharpness, contrast, etc.).In a non-limiting example, the position of the subordinate camera lensmay be adjusted to satisfy a threshold image metric (e.g., a thresholdsharpness, a threshold contrast, etc.) that indicates that the primarycamera is focused on an image subject. Additionally or alternatively,the subordinate camera may be focused on the image subjects based atleast in part on one or more autofocus techniques (e.g., phasedetection, contrast detection, laser autofocus, etc.).

At 308, the method 300 may include obtaining a second set of positiondata corresponding to respective focus positions of the subordinatecamera lens. For instance, the second set of position data may beobtained at least partly via one or more position sensors (e.g., theposition sensors 1208 described below with reference to FIG. 12 ) of thesubordinate camera. In some examples, the position sensors may includeone or more magnetic field sensors (e.g., Hall sensors, TMR sensors, GMRsensors, etc.). For instance, the subordinate camera may include a VCMactuator having one or more coils and one or more magnets, e.g., asdescribed below with reference to FIG. 12 . The coils may be configuredto receive a current and magnetically interact with one or more magnetsto produce Lorentz forces that cause the subordinate camera lens to movealong an optical axis. One or more position sensor magnets (e.g., theposition sensor magnets 1214 described below with reference to FIG. 12 )may be coupled to the subordinate camera lens such that the positionsensor magnets move along with the subordinate camera lens. One or moremagnetic field sensors may be used to detect the position of theposition sensor magnets, thereby enabling position sensing of thesubordinate camera lens.

At 310, the method 300 may include determining the focus relationshipbetween the primary camera and the subordinate camera based at least inpart on the first set of position data and the second set of positiondata. As discussed in further detail below with reference to FIGS. 7-11, in various embodiments, the focus relationship may be approximated aslinear. The nominal slope of the focus relationship may be based atleast in part on a focal length of the subordinate camera and a focallength of the primary camera. For instance, the nominal slope of thefocus relationship may be based at least in part on a ratio of the focallength of the subordinate camera to the focal length of the primarycamera. In various examples, the nominal slope of the focus relationshipmay be equal to the square of the ratio of the focal lengths.Furthermore, as discussed in further detail below with reference toFIGS. 6 and 8-11 , the focus relationship may include an offset term(also referred to herein as an “offset” or a “y-intercept”). The offsetterm may be the y-intercept of the linear relationship characterized bythe focus relationship. The offset term may change from time to timebased on one or more parameters associated with the primary cameraand/or the subordinate camera. In some instances, the focus relationshipmay be updated to account for a change in the parameter(s) that causes achange in the offset term of the focus relationship.

In some examples, the method 300 of determining the focus relationshipmay be performed during a calibration at a location of a manufacturer ofthe primary camera and/or the subordinate camera, and/or at a locationof a manufacturer of a product (e.g., a mobile multifunction device)that includes the primary camera and the subordinate camera.Additionally or alternatively, the method 300 may be performed at somepoint after the primary camera and the subordinate camera have reachedthe hands of a user (e.g., a user of a product that includes the primarycamera and the subordinate camera).

FIG. 4 is a flowchart of an example method 400 of updating a focusrelationship between a primary camera (e.g., the primary camera unit 102described above with reference to FIG. 1 ) and a subordinate camera(e.g., the subordinate camera unit 104 described above with reference toFIG. 1 ), in accordance with some embodiments.

At 402, the method 400 may include determining a focus relationshipbetween the primary camera and the subordinate camera. For instance, thefocus relationship may be determined using the method 300 describedabove with reference to FIG. 3 .

At 404, the method 400 may include focusing the subordinate camera basedat least in part on the focus relationship. For instance, thesubordinate camera lens may be moved to a focus position that isdetermined based at least in part on the focus relationship. In variousexamples, the subordinate camera may include an actuator to move thesubordinate camera lens along the optical axis and/or along a plane thatis orthogonal to the optical axis. In some examples, the actuator may bea voice coil motor (VCM) actuator, e.g., as illustrated below withreference to FIG. 12 . A controller may be used to determine the focusposition of the subordinate camera lens based at least in part on thefocus relationship, and cause a current to be supplied to one or morecoils of the VCM actuator based on the determined focus position. Thecurrent may cause the coils to magnetically interact with one or moreactuator magnets of the VCM actuator to produce Lorentz forces thatcause the coils or the actuator magnets to move. In some cases, thecoils or the actuator magnets are coupled (e.g., at least partly via alens holder) to the subordinate camera lens such that the subordinatecamera lens moves along with the coils or the actuator magnets. Thus,the supplied current may cause the subordinate camera lens to move tothe determined focus position via movement of the coils or the actuatormagnets.

In some examples, the controller may drive the primary camera lens focusposition with a first drive current, and drive the subordinate cameralens focus position with a second drive current that is based on thefirst drive current and the focus relationship. For instance, in somecases, the controller may not use feedback (e.g., position sensorfeedback corresponding to the primary camera lens focus position) todrive the subordinate camera lens focus position. In some embodiments,the drive current supplied to the primary camera and/or the subordinatecamera may be compensated for gravity.

At 406, the method 400 may include determining whether to update thefocus relationship. In some examples, the determination of whether toupdate the focus relationship may be based at least in part on a periodof time that has expired. For instance, a camera system (e.g., thecamera system 100 described above with reference to FIG. 1 ) thatincludes the primary camera and the subordinate camera may be configuredto update the focus relationship periodically. In some non-limitingexamples, the camera system may be configured to update the focusrelationship minute by minute, hourly, daily, weekly, monthly, and/oryearly. Additionally, or alternatively, the camera system may allow auser to input a desired time for updating the focus relationship and/ora desired frequency for updating the focus relationship.

In various implementations, the camera system may determine to updatethe focus relationship based at least in part on an operational state ofthe primary camera and/or the subordinate camera. For instance, thecamera system may detect that the primary camera and/or the subordinatecamera are currently not in use, or otherwise determine that the primarycamera and/or the subordinate camera will likely not be used within acertain time period. The camera system may determine to update the focusrelationship during such instances and/or during other instancesdetermined to be good opportunities for updating the focus relationshipwithout significantly negatively impacting user experience. In somecases, the camera system may determine to update the focus relationshipafter a user takes a picture using the camera system.

In some implementations, the camera system may determine to update thefocus relationship based at least in part on a confidence level of thefocus relationship. As discussed in further detail below with referenceto FIGS. 5 and 11 , the camera system may be configured to calculate aconfidence level of the focus relationship. The camera system maycompare the calculated confidence level to a threshold confidence level.The camera system may determine to update the focus relationship atleast partly responsive to determining that the calculated confidencelevel satisfies the threshold confidence level.

If, at 406, it is determined to update the focus relationship, then themethod 400 may include updating the focus relationship, at 408. Forexample, the focus relationship may be updated using the method 300described above with reference to FIG. 3 and/or the method 500 describedbelow with reference to FIG. 5 . If, at 406, it is determined to notupdate the focus relationship, then the method 400 may includecontinuing to focus the subordinate camera based at least in part on thecurrent focus relationship, at 404.

FIG. 5 is a flowchart of an example method 500 of determining an updatedfocus relationship between a primary camera (e.g., the primary cameraunit 102 described above with reference to FIG. 1 ) and a subordinatecamera (e.g., the subordinate camera unit 104 described above withreference to FIG. 1 ), in accordance with some embodiments. At 502, themethod 500 may include focusing the primary camera on at least one imagesubject. At 504, the method 500 may include obtaining a first set ofposition data corresponding to one or more focus positions of theprimary camera lens, e.g., as described above with reference to FIG. 3 .

At 506, the method 500 may include calculating a confidence level of thefocus relationship. At 508, the method 500 may include determining aconstrained focus range for the subordinate camera. For instance, theconstrained focus range may be determined based at least in part on theconfidence level of the focus relationship. At 510, the method 500 mayinclude focusing the subordinate camera on the image subject. Forinstance, subordinate camera may be focused on the image subject bymoving the subordinate camera lens within the constrained focus range.At 512, the method 500 may include obtaining a second set of positiondata corresponding to one or more focus positions of the subordinatecamera lens, e.g., as described above with reference to FIG. 3 .

At 514, the method 500 may include determining the second focusrelationship between the primary camera and the subordinate camera basedat least in part on the first set of position data and the second set ofposition data. As discussed in further detail below with reference toFIGS. 7-11 , in various embodiments, the focus relationship may beapproximated as linear. The nominal slope of the second focusrelationship may be based at least in part on a focal length of thesubordinate camera and a focal length of the primary camera. Forinstance, the nominal slope of the second focus relationship may bebased at least in part on a ratio of the focal length of the subordinatecamera to the focal length of the primary camera. In various examples,the nominal slope of the second focus relationship may be equal to thesquare of the ratio of the focal lengths. Furthermore, as discussed infurther detail below with reference to FIGS. 6 and 8-11 , the focusrelationship may include an offset term. The offset term may be they-intercept of the linear relationship characterized by the second focusrelationship. The offset term may change from time to time based on oneor more parameters associated with the primary camera and/or thesubordinate camera.

FIG. 6 is a flowchart of an example method 600 of updating an offsetterm of a focus relationship, in accordance with some embodiments. Forinstance, the method 600 may be performed as part of updating a focusrelationship, as described above with reference to FIG. 4 (e.g., atblock 408). The first offset term may be a part of the focusrelationship that is variable based at least in part on one or moreparameters corresponding to the primary camera and/or the subordinatecamera.

At 602, the method 600 may include determining a current state of one ormore parameters corresponding to the primary camera and/or thesubordinate camera. For instance, the parameters may be parameters thataffect focus positioning of the primary camera lens and/or thesubordinate camera lens, e.g., by causing effective focal length (EFL)variation. For example, the parameters may include temperaturesensitivity, ambient magnetic fields, long-term EFL variation-causingfactors (e.g., humidity), end stop compression in drop events, etc.

In some examples, the parameters may include a first temperatureassociated with the primary camera lens and a second temperatureassociated with the subordinate camera lens. The current state of thefirst temperature and the first state of the second temperature may beobtained, for example, via temperature measurements produced bytemperature sensors disposed at or near the primary camera lens and thesubordinate camera lens. In some examples, a first temperature sensormay be disposed such that it measures a temperature of a first componentdisposed near the primary lens. The temperature of the first componentmay be used as an approximation of the temperature of the primary lens.Similarly, a second temperature sensor may be disposed such that itmeasures a temperature of a second component disposed near thesubordinate lens. The temperature of the second component may be used asan approximation of the temperature of the subordinate lens. As lenstemperatures may change during operation of the primary camera and thesubordinate camera, the offset term of the focus relationship may alsochange.

At 604, the method 600 may include updating the offset term of the focusrelationship based at least in part on the current state of theparameters corresponding to the primary camera and/or the subordinatecamera. For instance, the offset term may be updated based at least inpart on the current state of the first temperature associated with theprimary camera lens and the current state of the second temperatureassociated with the subordinate camera lens. As discussed in furtherdetail below with reference to FIGS. 9 and 10 , temperature measurementsmay be used for determining and/or updating the focus relationshipbetween the primary camera and the subordinate camera based at least inpart on practical focal length (PFL) temperature compensation.

FIG. 7 is a flowchart of an example method 700 of maintaining focuscontinuity across a transition from one camera mode to another cameramode, in accordance with some embodiments. At 702, the method 700 mayinclude transitioning from a first camera mode to a second camera mode.Additionally, or alternatively, at 704, the method 700 may includetransitioning from the second camera mode to the first camera mode.

In the first camera mode, a first camera unit (e.g., the primary cameraunit 102 described above with reference to FIG. 1 ) may be designated asa primary camera for focusing, and a second camera unit (e.g., thesubordinate camera unit 104 described above with reference to FIG. 1 )may be designated as a subordinate camera for focusing. In the secondcamera mode, the second camera unit may be designated as the primarycamera for focusing, and the first camera unit may be designated as thesubordinate camera for focusing. That is, the first camera unit and thesecond camera unit may switch primary-subordinate roles from time totime. For instance, in some examples, the first camera unit and thesecond camera unit may switch primary-subordinate roles based at leastin part on camera settings information and/or ambient conditionsinformation (e.g., ambient lighting information). Additionally, oralternatively, a controller may determine to switch theprimary-subordinate roles of the first camera unit and the second cameraunit based at least in part on a respective camera lens type and/or arespective distance to a particular image subject. According to someembodiments, across transitioning from the first camera mode to thesecond camera mode and/or across transitioning from the second cameramode to the first camera mode, the primary camera (i.e., the camera unitdesignated as the primary camera) may be focused on an image subjectbased at least in part on image content corresponding to the imagesubject. Moreover, across such transitions, the subordinate camera(i.e., the camera unit designated as the subordinate camera may befocused on the image subject based at least in part on a focusrelationship between the first camera unit and the second camera unit.

At 706, the method 700 may include maintaining continuity of focus onthe image subject by the primary camera and the subordinate cameraacross the transition from the first camera mode to the second cameramode and/or across the transition from the second camera mode to thefirst camera mode. For instance, such continuity of focus acrosstransitions may be maintained by focusing the subordinate camera basedat least in part on a focus relationship between the subordinate cameraand the primary camera, as further described above and below withreference to FIGS. 1-6 and 8A-12 . In some cases, the focus relationshipmay be adjusted for a switch in the primary-subordinate roles of thefirst camera unit and the second camera unit. For instance, a firstfocus relationship may be used when the first camera unit is designatedas the primary camera and the second camera unit is designated as thesubordinate camera. A second focus relationship, which may correspond toan adjustment to the first focus relationship, may be used when thesecond camera unit is the primary camera and the first camera unit isthe subordinate camera. In some instances, the first focus relationshipand the second focus relationship may be determined before a switch inprimary-subordinate roles occurs. In other instances, an adjustment to afocus relationship to account for a switch in primary-subordinate rolesmay occur on-the-fly during a time period in which the switch occurs.

FIGS. 8A-8B illustrate, via graphs 800 a-800 c, example characteristicsof a focus relationship between positioning of a primary camera lens(e.g., the primary camera lens 106 of the primary camera unit 102illustrated in FIG. 1 ) and positioning of a subordinate camera lens(e.g., the subordinate camera lens 110 of the subordinate camera unit104 illustrated in FIG. 1 ), in accordance with some embodiments. Forexample, the focus relationship illustrated in the graphs of FIGS. 8A-8Bmay include a relationship between the primary camera lens andsubordinate camera lens focus positions when both cameras are in focuson a same object and at the same distance. In FIG. 8A, the left-sidegraph 800 a provides an example of positioning of the primary cameralens, and the right-side graph 800 b provides an example of positioningof the subordinate camera lens. In FIG. 8B, the graph 800 c provides anexample of a focus relationship line with respect to the primary cameralens position and the subordinate camera lens position.

In the left-side graph 800 a of FIG. 8A, the vertical axis, labeled“1/d_(o)”, represents an inverse of the distance from the primary cameralens to the object (or image subject). The horizontal axis, labeled“d_(i)”, represents a distance from the primary camera lens to an imagesensor of the primary camera. Moreover, the horizontal axis mayrepresent a practical focal length (PFL) of the primary camera. Invarious embodiments, the PFL of the primary camera may be determinedbased at least in part on measurements from one or more position sensors(e.g., the position sensors 1208 described below with reference to FIG.12 ). The graph 800 a indicates, via line 802, that there is a linearrelationship between the change in the PFL of the primary camera (or achange in d_(i)), denoted as Δ_(Primary), and a change in 1/d_(o),denoted as Δ(1/d_(o)). In some embodiments, the slope of the line 802may be characterized as:

${m_{Primary} \approx \frac{1}{EFL_{Primary}^{2}}},$

where:

m_(Primary) is the slope of the line 802, and

EFL_(Primary) is the effective focal length of primary camera lens.

In the right-side graph 800 b of FIG. 8A, the vertical axis, labeled“1/d_(o)”, represents an inverse of the distance from the subordinatecamera lens to the object (or image subject). The horizontal axis,labeled “d_(i)”, represents a distance from the subordinate camera lensto an image sensor of the subordinate camera. Moreover, the horizontalaxis may represent a practical focal length (PFL) of the subordinatecamera. In various embodiments, the PFL of the subordinate camera may bedetermined based at least in part on measurements from one or moreposition sensors (e.g., the position sensors 1208 described below withreference to FIG. 12 ). The graph 800 b indicates, via line 804, thatthere is a linear relationship between the change in the PFL of thesubordinate camera (or a change in d_(i)), denoted as “Δ_(Subordinate)”,and a change in 1/d_(o), denoted as “Δ(1/d_(o))”. In some embodiments,the slope of the line 804 may be characterized as follows:

${m_{Subordinate} \approx \frac{1}{EFL_{Subordinate}^{2}}},$

where:

m_(Subordinate) is the slope of the line 804, and

EFL_(Subordinate) is the effective focal length of subordinate cameralens.

Furthermore, the relationship between Δ_(Subordinate) and Δ_(Primary)may be characterized as follows:

${\Delta_{Subordinate} \approx {\Delta_{Primary}\left( \frac{EFL_{Subordinate}}{EFL_{Primary}} \right)}^{2}},$

In the graph 800 c of FIG. 8B, the vertical axis represents thesubordinate camera lens position and the vertical axis represents theprimary camera lens position. The graph indicates, via a focusrelationship line 806, that there is a linear relationship between focuspositioning of the subordinate camera lens and focus positioning of theprimary camera lens. In some examples, the focus relationship can beapproximated as linear due to the lens travel range being small comparedto the focal length of each camera. Furthermore, in some cases, theerrors associated with the linear approximation may be very small. Forinstance, in some cases the errors associated with the linearapproximation may be less than 1 micrometer over the travel ranges ofthe primary camera lens and the subordinate camera lens, which may besmaller than the uncertainty in some conventional autofocus techniques.

The nominal slope of the focus relationship line 806 may be based atleast in part on a focal length of the subordinate camera lens and afocal length of the primary camera lens. For instance, the nominal slopeof the focus relationship line 806 may be based at least in part on aratio of the focal length of the subordinate camera lens to the focallength of the primary camera lens. In various examples, the nominalslope of the focus relationship may be equal to the square of the ratioof the focal lengths, for example, characterized as follows:

${m_{{Focus}{Relationship}} = \left( \frac{EFL_{Subordinate}}{EFL_{Primary}} \right)^{2}},$

where m_(Focus Relationship) is the slope of the focus relationship line806.

The graph 800 c of FIG. 8B indicates, along the vertical axis, aninfinity end stop position of the subordinate camera lens, denoted as “∞Stop_(S)”, and an infinity focus position of the subordinate cameralens, denoted as “∞ Focus_(S)”. In addition, the graph 800 c indicates,along the vertical axis, a macro focus limit of the subordinate cameralens. Similarly, the graph 800 c indicates, along the horizontal axis,an infinity end stop position of the primary camera lens, denoted as “∞Stop_(P)”, and an infinity focus position of the primary camera lens,denoted as “∞ Focus_(P)”. In addition, the graph 800 c indicates, alongthe horizontal axis, a macro focus limit of the primary camera lens. Insome embodiments, the primary camera may be a wide angle lens camera andthe subordinate camera may be a telephoto camera. However, in otherembodiments, the primary camera may be a telephoto lens camera or anyother type of camera, and the subordinate camera may be a wide anglelens camera or any other type of camera.

The graph 800 c indicates a distance, denoted as “Z_(0,P)”, between thezero position of the primary camera lens and the infinity end stopposition of the primary camera lens (∞ Stop_(P)). Furthermore, the graph800 c indicates a distance, denoted as “ΔZ∞,_(P)”, between the infinityend stop position of the primary camera lens (∞ Stop_(P)) and theinfinity focus position of the primary camera (∞ Focus_(P)) lens.

Similarly, the graph 800 c indicates a distance, denoted as “Z_(0.S)”,between the zero position of the subordinate camera lens and theinfinity end stop position of the primary camera lens (∞ Stop_(P)).Furthermore, the graph 800 c indicates a distance, denoted as“ΔZ∞,_(S)”, between the infinity end stop position of the subordinatecamera lens (∞ Stop_(S)) and the infinity focus position of thesubordinate camera lens (∞ Focus_(S)).

The y-intercept of the focus relationship line 806 may represent anoffset term (e.g., the offset term described above with reference toFIGS. 3, 5, and 6 ) of the focus relationship. The offset term maychange from time to time based on one or more parameters associated withthe primary camera and/or the subordinate camera. In some instances, thefocus relationship may be updated to account for a change in theparameter(s) that causes a change in the offset term of the focusrelationship.

In some embodiments, the y-intercept of the focus relationship line 806may be determined based at least in part on distances between theinfinity focus positions and the infinity end stops, focus positionoffsets, and/or a ratio of focal lengths. In some examples, they-intercept of the focus relationship line 806 may be characterized asfollows:

${y{intercept}} = {\left( {{\Delta z_{\infty,S}} + {\Delta z_{0,S}}} \right) - {\left( {{\Delta z_{\infty,P}} + {\Delta z_{0,P}}} \right) \times {\left( \frac{EFL_{S}}{EFL_{P}} \right)^{2}.}}}$

The y-intercept position may depend on infinity end stop to infinityfocus spacing at a reference temperature, and may vary, e.g., based atleast in part on temperature sensitivity of the primary camera lensand/or the subordinate camera lens.

FIG. 9 is a block diagram of an example method 900 of focusing asubordinate camera (e.g., the subordinate camera unit 104 describedabove with reference to FIG. 1 ) using an adaptive lens model that isbased on a focus relationship between the subordinate camera and aprimary camera (e.g., the primary camera unit 102 described above withreference to FIG. 1 ), in accordance with some embodiments. In theexample method 900, a y-intercept estimator (e.g., the estimator 1100described below with reference to FIG. 11 ) may be used to update theadaptive lens model.

At 902, the method 900 may include focusing the primary camera on animage subject. In some examples, the primary camera unit may be focusedon the image subject based at least in part on image contentcorresponding to the image subject. For instance, focusing of theprimary camera unit on the image subject may include adjusting theposition of the primary camera lens based at least in part on one ormore metrics (e.g., sharpness, contrast, etc.). In a non-limitingexample, the position of the primary camera lens may be adjusted tosatisfy a threshold image metric (e.g., a threshold sharpness, athreshold contrast, etc.) that indicates that the primary camera unit isfocused on the image subject. Additionally or alternatively, the primarycamera unit may be focused on the image subject based at least in parton one or more autofocus techniques (e.g., phase detection, contrastdetection, laser autofocus, etc.). In some instances, the primary cameramay use focus pixels to maintain sharp focus in bright light.Furthermore, in some instances, the primary camera may search for afocus position using a contrast metric when focusing in low light. Oneor more position sensors (e.g., the position sensors 1208 describedbelow with reference to FIG. 12 ) may be used to determine the primarycamera lens position, which corresponds to a practical focal length(PFL) of the primary camera lens (denoted in FIG. 9 as “PFL_(P)”).

In some embodiments, a temperature of the primary camera lens (denotedin FIG. 9 as “T_(P)”) may be obtained during a time period in which theprimary camera is focused on the image subject. In some examples, thetemperature of the primary camera lens may be approximated as atemperature of another component of the primary camera that is near theprimary camera lens. Additionally, or alternatively, the temperature ofthe primary camera lens may be inferred based at least in part on avoice coil motor (VCM) actuator current and a thermal model.Additionally, or alternatively, the temperature of the primary cameralens may be derived from the VCM actuator coil resistance.

At 904, the method 900 may include performing an effective focal length(EFL) correction to account for changes in one or more parametersassociated with the primary camera lens and/or the primary camera. Forinstance, the EFL correction may include correcting for a variation inthe temperature of the primary camera lens, as further described belowwith reference to FIG. 10 . In some examples, the EFL correction mayproduce a temperature-corrected position of the primary camera lensbased at least in part on the PFL_(P) and the T_(P), which correspondsto a corrected/updated PFL of the primary camera lens (denoted in FIG. 9as “PFL_(P)′”).

At 906, the method 900 may include using the lens model to calculate,based at least in part on the focus relationship between the primarycamera and the subordinate camera, a corrected position of thesubordinate camera lens that corresponds to the same distance to theimage subject. The corrected position of the subordinate camera lens maycorrespond to a corrected PFL of the subordinate camera lens (denoted inFIG. 9 as “PFL_(S)′”). The lens model may be updated from time to timebased at least in part on y-intercept estimates provided by they-intercept estimator.

At 908, the method 900 may include performing an EFL correction toaccount for changes in one or more parameters associated with thesubordinate camera lens and/or the subordinate camera. For instance, theEFL correction may include correcting for a variation in the temperatureof the subordinate camera lens, as further described below withreference to FIG. 10 . In some examples, the EFL correction may producea temperature-corrected position of the subordinate camera lens based atleast in part on the PFL_(S)′ and a temperature of the subordinatecamera lens (denoted in FIG. 9 as “T_(S)”), which corresponds to atarget PFL of the subordinate camera lens (denoted in FIG. 9 as “PFL_(S)Target”).

At 910, the method 900 may include checking the target PFL of thesubordinate camera against a focus range of the subordinate camera. Forinstance, the subordinate camera may have a macro focus limit and aninfinity limit that define upper and lower bounds of the focus range ofthe subordinate camera, and the method 900 may include checking whetherthe target PFL of the subordinate camera is within the focus range. Insome embodiments, the y-intercept estimator may estimate the y-interceptof the focus relationship based at least in part on focus positions ofthe subordinate camera lens. When the target PFL of the subordinatecamera lens falls outside the focus range, the y-intercept estimator maynot use the corresponding position coordinates to update the focusrelationship as the subordinate camera may not be in focus.

At 912, the method 900 may include focusing the subordinate camera onthe same image subject as the primary camera. In some examples, thesubordinate camera unit may be focused on the image subject based atleast in part on image content corresponding to the image subject. Forinstance, focusing of the subordinate camera unit on the image subjectmay include adjusting the position of the subordinate camera lens basedat least in part on one or more metrics (e.g., sharpness, contrast,etc.). In a non-limiting example, the position of the subordinate cameralens may be adjusted to satisfy a threshold image metric (e.g., athreshold sharpness, a threshold contrast, etc.) that indicates that thesubordinate camera unit is focused on the image subject. Additionally oralternatively, the subordinate camera unit may be focused on the imagesubject based at least in part on one or more autofocus techniques(e.g., phase detection, contrast detection, laser autofocus, etc.). Insome instances, the subordinate camera may use focus pixels to maintainsharp focus in bright light. Furthermore, in some instances, thesubordinate camera may search for a focus position using a contrastmetric when focusing in low light. One or more position sensors (e.g.,the position sensors 1208 described below with reference to FIG. 12 )may be used to determine the subordinate camera lens position, whichcorresponds to a PFL of the subordinate camera lens (denoted in FIG. 9as “PFL_(S)”).

Additionally, or alternatively, the y-intercept estimator may provide asearch range within which to search for a focus position of thesubordinate camera lens. For instance, the search range may correspondto a constrained search within a subset of the focus range. In someexamples, the y-intercept estimator may determine the search range basedat least in part on a confidence level of the focus relationship. Incases where a constrained search is performed, the time to achieve thefocus position of the subordinate camera lens may be reduced as comparedto some conventional focus techniques that involve searching for a focusposition within the whole focus range.

In some embodiments, the subordinate camera may be focused on the imagesubject based at least in part on the focus relationship between thesubordinate camera and the primary camera. For instance, the subordinatecamera may track the target PFL of the subordinate camera lens (PFLsTarget) without searching for a focus position based on image contentassociated with the image subject.

Furthermore, at 912, a temperature of the subordinate camera lens(T_(S)) may be obtained during a time period in which the subordinatecamera is focused on the image subject. The temperature of thesubordinate camera lens may be used, e.g., in the EFL correctionperformed at 908 and/or in the EFL correction performed at 914.

At 914, the method 900 may include performing an EFL correction toaccount for changes in one or more parameters associated with thesubordinate camera lens and/or the subordinate camera. For instance, theEFL correction may include correcting for a variation in the temperatureof the subordinate camera lens, as further described below withreference to FIG. 10 . In some examples, the EFL correction may producea temperature-corrected position of the subordinate camera lens based atleast in part on the PFL_(S) and the T_(S), which corresponds to acorrected/updated PFL of the subordinate camera lens (denoted in FIG. 9as “PFL_(S)′”).

At 916, the method 900 may include estimating, by the y-interceptestimator, the y-intercept (also referred to herein as an “offset term”)of the focus relationship. The y-intercept estimator may estimate they-intercept based on one or more inputs. For instance, the corrected PFLof the primary camera lens (PFL_(P)′) and the corrected PFL of thesubordinate camera lens (PFL_(S)′) may be provided as inputs to they-intercept estimator. Accordingly, the y-intercept estimator mayestimate the y-intercept based at least in part on the PFL_(P)′ and thePFL_(S)′. Furthermore, the y-intercept estimator may receive, as inputs,information related to whether the primary camera and/or the subordinatecamera are in focus. Moreover, the y-intercept estimator may receive, asan input, information related to the range check performed at 910 (e.g.,whether the target PFL of the subordinate camera is within the focusrange).

Outlier values that correspond to situations where the primary cameraand the subordinate camera have focused on different image subjects maybe flagged and may not be included in the y-intercept estimatecalculated by the y-intercept estimator. Furthermore, values may also berejected when the primary camera and/or the subordinate camera has notconverged on focus (e.g., when focus peak is not found in a contrastfocus method) and/or when the subordinate camera has been directedoutside of its focus range.

FIG. 10 is a block diagram of an example method 1000 for calculating atemperature-corrected position of a primary camera lens (e.g., theprimary camera lens 106 of the primary camera unit 102 described abovewith reference to FIG. 1 ) and/or a subordinate camera lens (e.g., thesubordinate camera lens 110 of the subordinate camera unit 104 describedabove with reference to FIG. 1 ), in accordance with some embodiments.In some embodiments, the effective focal length (EFL) ratio used todetermine the focus relationship between the primary camera and thesubordinate camera may be based on the nominal EFLs recorded for each ofthe primary camera lens and the subordinate camera lens. These nominalEFLs may only be valid for the temperature at which the respectivenominal EFL was measured due to a temperature dependence of the EFL.Furthermore, based at least in part on the Lens Maker's Formula and anoffset relationship between a measured focus position of a lens and itsPFL, a temperature dependence of a measured focus position of a lens maybe characterized as follows:Z _(nom) =Z _(est)−∝_(T)(T _(lens) −T _(nom)),

where:

Z_(nom) is a temperature-corrected position of the lens,

Z_(est) is a measured focus position of the lens,

∝_(T) is an EFL temperature coefficient for the lens,

T_(lens) is a temperature of the lens obtained during a time period inwhich the focus position of the lens is measured, and

T_(nom) is a temperature of the lens at which the EFL of the lens wasrecorded.

At 1002, the method 1000 may include reading focused position dataassociated with a lens. The focused position data may include a focusedposition of the lens, Z_(est), which may be measured using one or moreposition sensors (e.g., the position sensors 1208 described below withreference to FIG. 12 ). Furthermore, the focused position data mayinclude a temperature of the lens obtained during a time period in whichthe focus position of the lens is measured, T_(lens).

At 1004, the method 1000 may include calculating a temperature-correctedposition of the lens, Z_(nom). For instance, as noted above, thetemperature-corrected position of the lens may be calculated using thefollowing equation:Z _(nom) =Z _(est)−∝_(T)(T _(lens) −T _(nom)).

At 1006, the method may include inputting the temperature-correctedposition into an estimator (e.g., the y-intercept estimator describedabove with reference to FIG. 9 and/or the estimator 1100 described belowwith reference to FIG. 11 ).

FIG. 11 is a block diagram of an example estimator 1100 for estimating afocus relationship between a primary camera (e.g., the primary cameraunit 102 described above with reference to FIG. 1 ) and a subordinatecamera (e.g., the subordinate camera unit 104 described above withreference to FIG. 1 ), in accordance with some embodiments.

In some embodiments the estimator 1100 may include estimating logic 1102that may be used to determine, calculate, and/or estimate the focusrelationship and/or the offset term (y-intercept) of the focusrelationship in one or more of the embodiments described herein withreference to FIGS. 1-11, 13, and 14 . Furthermore, in some examples, theestimator 1100 may include a confidence calculator 1104 for calculatingone or more confidence levels of the focus relationship and/or theoffset term of the focus relationship. Although the estimating logic1102 and the confidence calculator 1104 are shown as separate blocks inFIG. 11 , the confidence calculator 1104 may additionally oralternatively be included within the estimating logic 1102 and/or withinone or more other components of the estimator 1102.

In various examples, the estimator 1100 may receive one or more inputs.For instance, the estimator 1100 may receive a practical focal length(PFL) of the primary camera (denoted in FIG. 11 as “PFL_(P)”), a PFL ofthe subordinate camera (denoted in FIG. 11 as (denoted in FIG. 11 as“PFL_(P)”), time information, and/or temperature informationcorresponding to the primary camera lens and/or the subordinate cameralens. The estimator 1100 may determine an estimate 1106 of the focusrelationship based at least in part on the received inputs. Forinstance, the estimate 1106 may include an estimate of a slope (m) 1108of the focus relationship and/or an estimate of an offset term (b) 1110of the focus relationship. As a result of the estimate 1106, the focusrelationship 1112 between the primary camera and the subordinate cameramay be established. For instance, the focus relationship may relate thePFL of the subordinate camera lens (PFL_(S)) to the PFL of the primarycamera lens (PFL_(P)) based on the estimated slope (m) 1108 and theestimated offset term (b) 1110, as follows:PFL _(S) =m(PFL _(P))+b.

The estimator 1100 may update the focus relationship 1112 from time totime. For instance, the estimator 1100 may update the focus relationship1112 based at least in part on receiving one or more updated inputs.

FIG. 12 illustrates a schematic side view of an example camera modulehaving an example voice coil motor (VCM) actuator 1200 for moving anoptical package 1202, in accordance with some embodiments. In someembodiments, the example camera module may represent an example of aprimary camera (e.g., one or more of the primary camera embodimentsdescribed above with reference to FIGS. 1-11 ) and/or an example of asubordinate camera (e.g., one or more of the subordinate cameraembodiments described above with reference to FIGS. 1-11 ). However, itshould be understood that the primary camera and/or the subordinatecamera may include other camera architectures and/or actuatorarchitectures. As shown in FIG. 12 , the actuator 1200 may include abase or substrate 1204 and a cover 1206. The base 1204 may includeand/or support one or more position sensors (e.g., Hall sensors, TMRsensors, GMR sensors, etc.) 1208, one or more optical imagestabilization coils 1210, and one or more suspension wires 1212, whichmay at least partly enable magnetic sensing for autofocus and/or opticalimage stabilization position detection, e.g., by detecting movements ofposition sensor magnets 1214.

In some embodiments, the actuator 1200 may include one or more autofocuscoils 1216 and one or more actuator magnets 1218, which may at leastpartly enable autofocus functionality such as moving the optical package1202 along the z axis and/or along an optical axis defined by one ormore lenses of the optical package 1202. In some examples, at least oneposition sensor magnet 1214 may be disposed proximate to at least oneautofocus coil 1216. In some embodiments, at least one position sensormagnet 1214 may be coupled to at least one autofocus coil 1216. Forinstance, the autofocus coils 1216 may each define a central space thatis encircled by the respective autofocus coil 1216. The position sensormagnets 1214 may be disposed within the central spaces encircled by theautofocus coils 1216. Additionally or alternatively, the position sensormagnets 1214 may be attached to support structures (not shown) that arefixed to the autofocus coils 1216. For example, a support structure, towhich a position sensor magnet 1214 is attached, may be disposed withina central space encircled by an autofocus coil 1216 and the supportstructure may be fixed to the autofocus coil 1216.

In some embodiments, the actuator 1200 may include four suspension wires1212. The optical package 1202 may be suspended with respect to the base1204 by suspending one or more upper springs 1220 on the suspensionwires 1212. In some embodiments, the actuator may include one or morelower springs 1222. In the optical package 1202, an optics component(e.g., one or more lens elements, a lens assembly, etc.) may be screwed,mounted or otherwise held in or by an optics holder. Note that upperspring(s) 1220 and lower spring(s) 1222 may be flexible to allow theoptical package 1202 a range of motion along the Z (optical) axis foroptical focusing, and suspension wires 1212 may be flexible to allow arange of motion on the x-y plane orthogonal to the optical axis foroptical image stabilization. Also note that, while embodiments show theoptical package 1202 suspended on wires 1212, other mechanisms may beused to suspend the optical package 1202 in other embodiments.

In various embodiments, the camera module may include an image sensor1224. The image sensor 1224 may be disposed below the optical package1202 such that light rays may pass through one or more lens elements ofthe optical package 1202 (e.g., via an aperture at the top of theoptical package 1202) and to the image sensor 1224.

Multifunction Device Examples

Embodiments of electronic devices, user interfaces for such devices, andassociated processes for using such devices are described. In someembodiments, the device is a portable communications device, such as amobile telephone, that also contains other functions, such as PDA and/ormusic player functions. Example embodiments of portable multifunctiondevices include, without limitation, the iPhone®, iPod Touch®, and iPad®devices from Apple Inc. of Cupertino, Calif. Other portable electronicdevices, such as laptops, cameras, cell phones, or tablet computers, mayalso be used. It should also be understood that, in some embodiments,the device is not a portable communications device, but is a desktopcomputer with a camera. In some embodiments, the device is a gamingcomputer with orientation sensors (e.g., orientation sensors in a gamingcontroller). In other embodiments, the device is not a portablecommunications device, but is a camera.

In the discussion that follows, an electronic device that includes adisplay and a touch-sensitive surface is described. It should beunderstood, however, that the electronic device may include one or moreother physical user-interface devices, such as a physical keyboard, amouse and/or a joystick.

The device typically supports a variety of applications, such as one ormore of the following: a drawing application, a presentationapplication, a word processing application, a website creationapplication, a disk authoring application, a spreadsheet application, agaming application, a telephone application, a video conferencingapplication, an e-mail application, an instant messaging application, aworkout support application, a photo management application, a digitalcamera application, a digital video camera application, a web browsingapplication, a digital music player application, and/or a digital videoplayer application.

The various applications that may be executed on the device may use atleast one common physical user-interface device, such as thetouch-sensitive surface. One or more functions of the touch-sensitivesurface as well as corresponding information displayed on the device maybe adjusted and/or varied from one application to the next and/or withina respective application. In this way, a common physical architecture(such as the touch-sensitive surface) of the device may support thevariety of applications with user interfaces that are intuitive andtransparent to the user.

Attention is now directed toward embodiments of portable devices withcameras. FIG. 13 illustrates a block diagram of an example portablemultifunction device 1300 that may include a primary camera (e.g., theprimary camera unit 102 illustrated in FIG. 1 ) and a subordinate camera(e.g., the subordinate camera unit 104 illustrated in FIG. 1 ), inaccordance with some embodiments. Cameras 1364 are sometimes called“optical sensors” for convenience, and may also be known as or called anoptical sensor system. Device 1300 may include memory 1302 (which mayinclude one or more computer readable storage mediums), memorycontroller 1322, one or more processing units (CPUs) 1320, peripheralsinterface 1318, RF circuitry 1308, audio circuitry 1310, speaker 1311,touch-sensitive display system 1312, microphone 1313, input/output (I/O)subsystem 1306, other input or control devices 1316, and external port1324. Device 1300 may include multiple optical sensors 1364 (e.g., theprimary camera unit 102 and the subordinate camera unit 104 illustratedin FIG. 1 ). These components may communicate over one or morecommunication buses or signal lines 1303.

It should be appreciated that device 1300 is only one example of aportable multifunction device, and that device 1300 may have more orfewer components than shown, may combine two or more components, or mayhave a different configuration or arrangement of the components. Thevarious components shown in FIG. 13 may be implemented in hardware,software, or a combination of hardware and software, including one ormore signal processing and/or application specific integrated circuits.

Memory 1302 may include high-speed random access memory and may alsoinclude non-volatile memory, such as one or more magnetic disk storagedevices, flash memory devices, or other non-volatile solid-state memorydevices. Access to memory 1302 by other components of device 1300, suchas CPU 1320 and the peripherals interface 1318, may be controlled bymemory controller 1322.

Peripherals interface 1318 can be used to couple input and outputperipherals of the device to CPU 1320 and memory 1302. The one or moreprocessors 1320 run or execute various software programs and/or sets ofinstructions stored in memory 1302 to perform various functions fordevice 1300 and to process data.

In some embodiments, peripherals interface 1318, CPU 1320, and memorycontroller 1322 may be implemented on a single chip, such as chip 1304.In some other embodiments, they may be implemented on separate chips.

RF (radio frequency) circuitry 1308 receives and sends RF signals, alsocalled electromagnetic signals. RF circuitry 1308 converts electricalsignals to/from electromagnetic signals and communicates withcommunications networks and other communications devices via theelectromagnetic signals. RF circuitry 1308 may include well-knowncircuitry for performing these functions, including but not limited toan antenna system, an RF transceiver, one or more amplifiers, a tuner,one or more oscillators, a digital signal processor, a CODEC chipset, asubscriber identity module (SIM) card, memory, and so forth. RFcircuitry 1308 may communicate with networks, such as the Internet, alsoreferred to as the World Wide Web (WWW), an intranet and/or a wirelessnetwork, such as a cellular telephone network, a wireless local areanetwork (LAN) and/or a metropolitan area network (MAN), and otherdevices by wireless communication. The wireless communication may useany of a variety of communications standards, protocols andtechnologies, including but not limited to Global System for MobileCommunications (GSM), Enhanced Data GSM Environment (EDGE), high-speeddownlink packet access (HSDPA), high-speed uplink packet access (HSUPA),wideband code division multiple access (W-CDMA), code division multipleaccess (CDMA), time division multiple access (TDMA), Bluetooth, WirelessFidelity (Wi-Fi) (e.g., IEEE 802.11a, IEEE 802.11b, IEEE 802.11g and/orIEEE 802.11n), voice over Internet Protocol (VoIP), Wi-MAX, a protocolfor e-mail (e.g., Internet message access protocol (IMAP) and/or postoffice protocol (POP)), instant messaging (e.g., extensible messagingand presence protocol (XMPP), Session Initiation Protocol for InstantMessaging and Presence Leveraging Extensions (SIMPLE), Instant Messagingand Presence Service (IMPS)), and/or Short Message Service (SMS), or anyother suitable communication protocol, including communication protocolsnot yet developed as of the filing date of this document.

Audio circuitry 1310, speaker 1311, and microphone 1313 provide an audiointerface between a user and device 1300. Audio circuitry 1310 receivesaudio data from peripherals interface 1318, converts the audio data toan electrical signal, and transmits the electrical signal to speaker1311. Speaker 1311 converts the electrical signal to human-audible soundwaves. Audio circuitry 1310 also receives electrical signals convertedby microphone 1313 from sound waves. Audio circuitry 1310 converts theelectrical signal to audio data and transmits the audio data toperipherals interface 1318 for processing. Audio data may be retrievedfrom and/or transmitted to memory 1302 and/or RF circuitry 1308 byperipherals interface 1318. In some embodiments, audio circuitry 1310also includes a headset jack (e.g., 1412, FIG. 14 ). The headset jackprovides an interface between audio circuitry 1310 and removable audioinput/output peripherals, such as output-only headphones or a headsetwith both output (e.g., a headphone for one or both ears) and input(e.g., a microphone).

I/O subsystem 1306 couples input/output peripherals on device 1300, suchas touch screen 1312 and other input control devices 1316, toperipherals interface 1318. I/O subsystem 1306 may include displaycontroller 1356 and one or more input controllers 1360 for other inputor control devices. The one or more input controllers 1360 receive/sendelectrical signals from/to other input or control devices 1316. Theother input control devices 1316 may include physical buttons (e.g.,push buttons, rocker buttons, etc.), dials, slider switches, joysticks,click wheels, and so forth. In some alternate embodiments, inputcontroller(s) 1360 may be coupled to any (or none) of the following: akeyboard, infrared port, USB port, and a pointer device such as a mouse.The one or more buttons (e.g., 1408, FIG. 14 ) may include an up/downbutton for volume control of speaker 1311 and/or microphone 1313. Theone or more buttons may include a push button (e.g., 1406, FIG. 14 ).

Touch-sensitive display 1312 provides an input interface and an outputinterface between the device and a user. Display controller 1356receives and/or sends electrical signals from/to touch screen 1312.Touch screen 1312 displays visual output to the user. The visual outputmay include graphics, text, icons, video, and any combination thereof(collectively termed “graphics”). In some embodiments, some or all ofthe visual output may correspond to user-interface objects.

Touch screen 1312 has a touch-sensitive surface, sensor or set ofsensors that accepts input from the user based on haptic and/or tactilecontact. Touch screen 1312 and display controller 1356 (along with anyassociated modules and/or sets of instructions in memory 1302) detectcontact (and any movement or breaking of the contact) on touch screen1312 and converts the detected contact into interaction withuser-interface objects (e.g., one or more soft keys, icons, web pages orimages) that are displayed on touch screen 1312. In an exampleembodiment, a point of contact between touch screen 1312 and the usercorresponds to a finger of the user.

Touch screen 1312 may use LCD (liquid crystal display) technology, LPD(light emitting polymer display) technology, or LED (light emittingdiode) technology, although other display technologies may be used inother embodiments. Touch screen 1312 and display controller 1356 maydetect contact and any movement or breaking thereof using any of avariety of touch sensing technologies now known or later developed,including but not limited to capacitive, resistive, infrared, andsurface acoustic wave technologies, as well as other proximity sensorarrays or other elements for determining one or more points of contactwith touch screen 1312. In an example embodiment, projected mutualcapacitance sensing technology is used, such as that found in theiPhone®, iPod Touch®, and iPad® from Apple Inc. of Cupertino, Calif.

Touch screen 1312 may have a video resolution in excess of 800 dpi. Insome embodiments, the touch screen has a video resolution ofapproximately 860 dpi. The user may make contact with touch screen 1312using any suitable object or appendage, such as a stylus, a finger, andso forth. In some embodiments, the user interface is designed to workprimarily with finger-based contacts and gestures, which can be lessprecise than stylus-based input due to the larger area of contact of afinger on the touch screen. In some embodiments, the device translatesthe rough finger-based input into a precise pointer/cursor position orcommand for performing the actions desired by the user.

In some embodiments, in addition to the touch screen, device 1300 mayinclude a touchpad (not shown) for activating or deactivating particularfunctions. In some embodiments, the touchpad is a touch-sensitive areaof the device that, unlike the touch screen, does not display visualoutput. The touchpad may be a touch-sensitive surface that is separatefrom touch screen 1312 or an extension of the touch-sensitive surfaceformed by the touch screen.

Device 1300 also includes power system 1362 for powering the variouscomponents. Power system 1362 may include a power management system, oneor more power sources (e.g., battery, alternating current (AC)), arecharging system, a power failure detection circuit, a power converteror inverter, a power status indicator (e.g., a light-emitting diode(LED)) and any other components associated with the generation,management and distribution of power in portable devices.

Device 1300 may also include one or more optical sensors or cameras1364. FIG. 13 shows an optical sensor 1364 coupled to optical sensorcontroller 1358 in I/O subsystem 1306. Optical sensor 1364 may includecharge-coupled device (CCD) or complementary metal-oxide semiconductor(CMOS) phototransistors. Optical sensor 1364 receives light from theenvironment, projected through one or more lens, and converts the lightto data representing an image. In conjunction with imaging module 1343(also called a camera module), optical sensor 1364 may capture stillimages or video. In some embodiments, an optical sensor 1364 is locatedon the back of device 1300, opposite touch screen display 1312 on thefront of the device, so that the touch screen display 1312 may be usedas a viewfinder for still and/or video image acquisition. In someembodiments, another optical sensor is located on the front of thedevice so that the user's image may be obtained for videoconferencingwhile the user views the other video conference participants on thetouch screen display.

Device 1300 may also include one or more proximity sensors 1366. FIG. 13shows proximity sensor 1366 coupled to peripherals interface 1318.Alternately, proximity sensor 1366 may be coupled to input controller1360 in I/O subsystem 1306. In some embodiments, the proximity sensor1366 turns off and disables touch screen 1312 when the multifunctiondevice 1300 is placed near the user's ear (e.g., when the user is makinga phone call).

Device 1300 includes one or more orientation sensors 1368. In someembodiments, the one or more orientation sensors 1368 include one ormore accelerometers (e.g., one or more linear accelerometers and/or oneor more rotational accelerometers). In some embodiments, the one or moreorientation sensors 1368 include one or more gyroscopes. In someembodiments, the one or more orientation sensors 1368 include one ormore magnetometers. In some embodiments, the one or more orientationsensors 1368 include one or more of global positioning system (GPS),Global Navigation Satellite System (GLONASS), and/or other globalnavigation system receivers. The GPS, GLONASS, and/or other globalnavigation system receivers may be used for obtaining informationconcerning the location and orientation (e.g., portrait or landscape) ofdevice 1300. In some embodiments, the one or more orientation sensors1368 include any combination of orientation/rotation sensors. FIG. 13shows the one or more orientation sensors 1368 coupled to peripheralsinterface 1318. Alternately, the one or more orientation sensors 1368may be coupled to an input controller 1360 in I/O subsystem 1306. Insome embodiments, information is displayed on the touch screen display1312 in a portrait view or a landscape view based on an analysis of datareceived from the one or more orientation sensors 1368.

In some embodiments, the software components stored in memory 1302include operating system 1326, communication module (or set ofinstructions) 1328, contact/motion module (or set of instructions) 1330,graphics module (or set of instructions) 1332, text input module (or setof instructions) 1334, Global Positioning System (GPS) module (or set ofinstructions) 1335, arbiter module 1358 and applications (or sets ofinstructions) 1336. Furthermore, in some embodiments memory 1302 storesdevice/global internal state 1357. Device/global internal state 1357includes one or more of: active application state, indicating whichapplications, if any, are currently active; display state, indicatingwhat applications, views or other information occupy various regions oftouch screen display 1312; sensor state, including information obtainedfrom the device's various sensors and input control devices 1316; andlocation information concerning the device's location and/or attitude.

Operating system 1326 (e.g., Darwin, RTXC, LINUX, UNIX, OS X, WINDOWS,or an embedded operating system such as VxWorks) includes varioussoftware components and/or drivers for controlling and managing generalsystem tasks (e.g., memory management, storage device control, powermanagement, etc.) and facilitates communication between various hardwareand software components.

Communication module 1328 facilitates communication with other devicesover one or more external ports 1324 and also includes various softwarecomponents for handling data received by RF circuitry 1308 and/orexternal port 1324. External port 1324 (e.g., Universal Serial Bus(USB), FIREWIRE, etc.) is adapted for coupling directly to other devicesor indirectly over a network (e.g., the Internet, wireless LAN, etc.).In some embodiments, the external port is a multi-pin (e.g., 30-pin)connector.

Contact/motion module 1330 may detect contact with touch screen 1312 (inconjunction with display controller 1356) and other touch sensitivedevices (e.g., a touchpad or physical click wheel). Contact/motionmodule 1330 includes various software components for performing variousoperations related to detection of contact, such as determining ifcontact has occurred (e.g., detecting a finger-down event), determiningif there is movement of the contact and tracking the movement across thetouch-sensitive surface (e.g., detecting one or more finger-draggingevents), and determining if the contact has ceased (e.g., detecting afinger-up event or a break in contact). Contact/motion module 1330receives contact data from the touch-sensitive surface. Determiningmovement of the point of contact, which is represented by a series ofcontact data, may include determining speed (magnitude), velocity(magnitude and direction), and/or an acceleration (a change in magnitudeand/or direction) of the point of contact. These operations may beapplied to single contacts (e.g., one finger contacts) or to multiplesimultaneous contacts (e.g., “multitouch”/multiple finger contacts). Insome embodiments, contact/motion module 1330 and display controller 1356detect contact on a touchpad.

Contact/motion module 1330 may detect a gesture input by a user.Different gestures on the touch-sensitive surface have different contactpatterns. Thus, a gesture may be detected by detecting a particularcontact pattern. For example, detecting a finger tap gesture includesdetecting a finger-down event followed by detecting a finger-up (liftoff) event at the same position (or substantially the same position) asthe finger-down event (e.g., at the position of an icon). As anotherexample, detecting a finger swipe gesture on the touch-sensitive surfaceincludes detecting a finger-down event followed by detecting one or morefinger-dragging events, and subsequently followed by detecting afinger-up (lift off) event.

Graphics module 1332 includes various known software components forrendering and displaying graphics on touch screen 1312 or other display,including components for changing the intensity of graphics that aredisplayed. As used herein, the term “graphics” includes any object thatcan be displayed to a user, including without limitation text, webpages, icons (such as user-interface objects including soft keys),digital images, videos, animations and the like.

In some embodiments, graphics module 1332 stores data representinggraphics to be used. Each graphic may be assigned a corresponding code.Graphics module 1332 receives, from applications etc., one or more codesspecifying graphics to be displayed along with, if necessary, coordinatedata and other graphic property data, and then generates screen imagedata to output to display controller 1356.

Text input module 1334, which may be a component of graphics module1332, provides soft keyboards for entering text in various applications(e.g., contacts 1337, e-mail 1340, IM 1341, browser 1347, and any otherapplication that needs text input).

GPS module 1335 determines the location of the device and provides thisinformation for use in various applications (e.g., to telephone 1338 foruse in location-based dialing, to camera 1343 as picture/video metadata,and to applications that provide location-based services such as weatherwidgets, local yellow page widgets, and map/navigation widgets).

Applications 1336 may include the following modules (or sets ofinstructions), or a subset or superset thereof:

-   -   contacts module 1337 (sometimes called an address book or        contact list);    -   telephone module 1338;    -   video conferencing module 1339;    -   e-mail client module 1340;    -   instant messaging (IM) module 1341;    -   workout support module 1342;    -   camera module 1343 for still and/or video images;    -   image management module 1344;    -   browser module 1347;    -   calendar module 1348;    -   widget modules 1349, which may include one or more of: weather        widget 1349-1, stocks widget 1349-2, calculator widget 1349-3,        alarm clock widget 1349-4, dictionary widget 1349-5, and other        widgets obtained by the user, as well as user-created widgets        1349-6;    -   widget creator module 1350 for making user-created widgets        1349-6;    -   search module 1351;    -   video and music player module 1352, which may be made up of a        video player module and a music player module;    -   notes module 1353;    -   map module 1354; and/or    -   online video module 1355.

Examples of other applications 1336 that may be stored in memory 1302include other word processing applications, other image editingapplications, drawing applications, presentation applications,JAVA-enabled applications, encryption, digital rights management, voicerecognition, and voice replication.

In conjunction with touch screen 1312, display controller 1356, contactmodule 1330, graphics module 1332, and text input module 1334, contactsmodule 1337 may be used to manage an address book or contact list (e.g.,stored in application internal state 1357), including: adding name(s) tothe address book; deleting name(s) from the address book; associatingtelephone number(s), e-mail address(es), physical address(es) or otherinformation with a name; associating an image with a name; categorizingand sorting names; providing telephone numbers or e-mail addresses toinitiate and/or facilitate communications by telephone 1338, videoconference 1339, e-mail 1340, or IM 1341; and so forth.

In conjunction with RF circuitry 1308, audio circuitry 1310, speaker1311, microphone 1313, touch screen 1312, display controller 1356,contact module 1330, graphics module 1332, and text input module 1334,telephone module 1338 may be used to enter a sequence of characterscorresponding to a telephone number, access one or more telephonenumbers in address book 1337, modify a telephone number that has beenentered, dial a respective telephone number, conduct a conversation anddisconnect or hang up when the conversation is completed. As notedabove, the wireless communication may use any of a variety ofcommunications standards, protocols and technologies.

In conjunction with RF circuitry 1308, audio circuitry 1310, speaker1311, microphone 1313, touch screen 1312, display controller 1356,optical sensor 1364, optical sensor controller 1358, contact module1330, graphics module 1332, text input module 1334, contact list 1337,and telephone module 1338, videoconferencing module 1339 includesexecutable instructions to initiate, conduct, and terminate a videoconference between a user and one or more other participants inaccordance with user instructions.

In conjunction with RF circuitry 1308, touch screen 1312, displaycontroller 1356, contact module 1330, graphics module 1332, and textinput module 1334, e-mail client module 1340 includes executableinstructions to create, send, receive, and manage e-mail in response touser instructions. In conjunction with image management module 1344,e-mail client module 1340 makes it very easy to create and send e-mailswith still or video images taken with camera module 1343.

In conjunction with RF circuitry 1308, touch screen 1312, displaycontroller 1356, contact module 1330, graphics module 1332, and textinput module 1334, the instant messaging module 1341 includes executableinstructions to enter a sequence of characters corresponding to aninstant message, to modify previously entered characters, to transmit arespective instant message (for example, using a Short Message Service(SMS) or Multimedia Message Service (MMS) protocol for telephony-basedinstant messages or using XMPP, SIMPLE, or IMPS for Internet-basedinstant messages), to receive instant messages and to view receivedinstant messages. In some embodiments, transmitted and/or receivedinstant messages may include graphics, photos, audio files, video filesand/or other attachments as are supported in a MMS and/or an EnhancedMessaging Service (EMS). As used herein, “instant messaging” refers toboth telephony-based messages (e.g., messages sent using SMS or MMS) andInternet-based messages (e.g., messages sent using XMPP, SIMPLE, orIMPS).

In conjunction with RF circuitry 1308, touch screen 1312, displaycontroller 1356, contact module 1330, graphics module 1332, text inputmodule 1334, GPS module 1335, map module 1354, and music player module1346, workout support module 1342 includes executable instructions tocreate workouts (e.g., with time, distance, and/or calorie burninggoals); communicate with workout sensors (sports devices); receiveworkout sensor data; calibrate sensors used to monitor a workout; selectand play music for a workout; and display, store and transmit workoutdata.

In conjunction with touch screen 1312, display controller 1356, opticalsensor(s) 1364, optical sensor controller 1358, contact module 1330,graphics module 1332, and image management module 1344, camera module1343 includes executable instructions to capture still images or video(including a video stream) and store them into memory 1302, modifycharacteristics of a still image or video, or delete a still image orvideo from memory 1302.

In conjunction with touch screen 1312, display controller 1356, contactmodule 1330, graphics module 1332, text input module 1334, and cameramodule 1343, image management module 1344 includes executableinstructions to arrange, modify (e.g., edit), or otherwise manipulate,label, delete, present (e.g., in a digital slide show or album), andstore still and/or video images.

In conjunction with RF circuitry 1308, touch screen 1312, display systemcontroller 1356, contact module 1330, graphics module 1332, and textinput module 1334, browser module 1347 includes executable instructionsto browse the Internet in accordance with user instructions, includingsearching, linking to, receiving, and displaying web pages or portionsthereof, as well as attachments and other files linked to web pages.

In conjunction with RF circuitry 1308, touch screen 1312, display systemcontroller 1356, contact module 1330, graphics module 1332, text inputmodule 1334, e-mail client module 1340, and browser module 1347,calendar module 1348 includes executable instructions to create,display, modify, and store calendars and data associated with calendars(e.g., calendar entries, to do lists, etc.) in accordance with userinstructions.

In conjunction with RF circuitry 1308, touch screen 1312, display systemcontroller 1356, contact module 1330, graphics module 1332, text inputmodule 1334, and browser module 1347, widget modules 1349 aremini-applications that may be downloaded and used by a user (e.g.,weather widget 549-1, stocks widget 549-2, calculator widget 13493,alarm clock widget 1349-4, and dictionary widget 1349-5) or created bythe user (e.g., user-created widget 1349-6). In some embodiments, awidget includes an HTML (Hypertext Markup Language) file, a CSS(Cascading Style Sheets) file, and a JavaScript file. In someembodiments, a widget includes an XML (Extensible Markup Language) fileand a JavaScript file (e.g., Yahoo! Widgets).

In conjunction with RF circuitry 1308, touch screen 1312, display systemcontroller 1356, contact module 1330, graphics module 1332, text inputmodule 1334, and browser module 1347, the widget creator module 1350 maybe used by a user to create widgets (e.g., turning a user-specifiedportion of a web page into a widget).

In conjunction with touch screen 1312, display system controller 1356,contact module 1330, graphics module 1332, and text input module 1334,search module 1351 includes executable instructions to search for text,music, sound, image, video, and/or other files in memory 1302 that matchone or more search criteria (e.g., one or more user-specified searchterms) in accordance with user instructions.

In conjunction with touch screen 1312, display system controller 1356,contact module 1330, graphics module 1332, audio circuitry 1310, speaker1311, RF circuitry 1308, and browser module 1347, video and music playermodule 1352 includes executable instructions that allow the user todownload and play back recorded music and other sound files stored inone or more file formats, such as MP3 or AAC files, and executableinstructions to display, present or otherwise play back videos (e.g., ontouch screen 1312 or on an external, connected display via external port1324). In some embodiments, device 1300 may include the functionality ofan MP3 player.

In conjunction with touch screen 1312, display controller 1356, contactmodule 1330, graphics module 1332, and text input module 1334, notesmodule 1353 includes executable instructions to create and manage notes,to do lists, and the like in accordance with user instructions.

In conjunction with RF circuitry 1308, touch screen 1312, display systemcontroller 1356, contact module 1330, graphics module 1332, text inputmodule 1334, GPS module 1335, and browser module 1347, map module 1354may be used to receive, display, modify, and store maps and dataassociated with maps (e.g., driving directions; data on stores and otherpoints of interest at or near a particular location; and otherlocation-based data) in accordance with user instructions.

In conjunction with touch screen 1312, display system controller 1356,contact module 1330, graphics module 1332, audio circuitry 1310, speaker1311, RF circuitry 1308, text input module 1334, e-mail client module1340, and browser module 1347, online video module 1355 includesinstructions that allow the user to access, browse, receive (e.g., bystreaming and/or download), play back (e.g., on the touch screen or onan external, connected display via external port 1324), send an e-mailwith a link to a particular online video, and otherwise manage onlinevideos in one or more file formats, such as H.264. In some embodiments,instant messaging module 1341, rather than e-mail client module 1340, isused to send a link to a particular online video.

Each of the above identified modules and applications correspond to aset of executable instructions for performing one or more functionsdescribed above and the methods described in this application (e.g., thecomputer-implemented methods and other information processing methodsdescribed herein). These modules (i.e., sets of instructions) need notbe implemented as separate software programs, procedures or modules, andthus various subsets of these modules may be combined or otherwisere-arranged in various embodiments. In some embodiments, memory 1302 maystore a subset of the modules and data structures identified above.Furthermore, memory 1302 may store additional modules and datastructures not described above.

In some embodiments, device 1300 is a device where operation of apredefined set of functions on the device is performed exclusivelythrough a touch screen and/or a touchpad. By using a touch screen and/ora touchpad as the primary input control device for operation of device1300, the number of physical input control devices (such as pushbuttons, dials, and the like) on device 1300 may be reduced.

The predefined set of functions that may be performed exclusivelythrough a touch screen and/or a touchpad include navigation between userinterfaces. In some embodiments, the touchpad, when touched by the user,navigates device 1300 to a main, home, or root menu from any userinterface that may be displayed on device 1300. In such embodiments, thetouchpad may be referred to as a “menu button.” In some otherembodiments, the menu button may be a physical push button or otherphysical input control device instead of a touchpad.

FIG. 14 depicts illustrates an example portable multifunction device1300 that may include a primary camera (e.g., the primary camera unit102 illustrated in FIG. 1 ) and a subordinate camera (e.g., thesubordinate camera unit 104 illustrated in FIG. 1 ), in accordance withsome embodiments. The device 1300 may have a touch screen 1312. Thetouch screen 1312 may display one or more graphics within user interface(UI) 1400. In this embodiment, as well as others described below, a usermay select one or more of the graphics by making a gesture on thegraphics, for example, with one or more fingers 1402 (not drawn to scalein the figure) or one or more styluses 1403 (not drawn to scale in thefigure).

Device 1300 may also include one or more physical buttons, such as“home” or menu button 1404. As described previously, menu button 1404may be used to navigate to any application 1336 in a set of applicationsthat may be executed on device 1300. Alternatively, in some embodiments,the menu button 1404 is implemented as a soft key in a GUI displayed ontouch screen 1312.

In one embodiment, device 1300 includes touch screen 1312, menu button1404, push button 1406 for powering the device on/off and locking thedevice, volume adjustment button(s) 1408, Subscriber Identity Module(SIM) card slot 1410, head set jack 1412, and docking/charging externalport 1324. Push button 1406 may be used to turn the power on/off on thedevice by depressing the button and holding the button in the depressedstate for a predefined time interval; to lock the device by depressingthe button and releasing the button before the predefined time intervalhas elapsed; and/or to unlock the device or initiate an unlock process.In an alternative embodiment, device 1300 also may accept verbal inputfor activation or deactivation of some functions through microphone1313.

It should be noted that, although many of the examples herein are givenwith reference to optical sensor(s)/camera(s) 1364 (on the front of adevice), one or more rear-facing cameras or optical sensors that arepointed opposite from the display may be used instead of, or in additionto, an optical sensor(s)/camera(s) 1364 on the front of a device.

Example Computer System

FIG. 15 illustrates an example computer system 1500 that may include aprimary camera (e.g., the primary camera unit 102 illustrated in FIG. 1) and a subordinate camera (e.g., the subordinate camera unit 104illustrated in FIG. 1 ), according to some embodiments. The computersystem 1500 may be configured to execute any or all of the embodimentsdescribed above. In different embodiments, computer system 1500 may beany of various types of devices, including, but not limited to, apersonal computer system, desktop computer, laptop, notebook, tablet,slate, pad, or netbook computer, mainframe computer system, handheldcomputer, workstation, network computer, a camera, a set top box, amobile device, a consumer device, video game console, handheld videogame device, application server, storage device, a television, a videorecording device, a peripheral device such as a switch, modem, router,or in general any type of computing or electronic device.

Various embodiments of a camera motion control system as describedherein, including embodiments of magnetic position sensing, as describedherein may be executed in one or more computer systems 1500, which mayinteract with various other devices. Note that any component, action, orfunctionality described above with respect to FIGS. 1-14 may beimplemented on one or more computers configured as computer system 1500of FIG. 15 , according to various embodiments. In the illustratedembodiment, computer system 1500 includes one or more processors 1510coupled to a system memory 1520 via an input/output (I/O) interface1530. Computer system 1500 further includes a network interface 1540coupled to I/O interface 1530, and one or more input/output devices1550, such as cursor control device 1560, keyboard 1570, and display(s)1580. In some cases, it is contemplated that embodiments may beimplemented using a single instance of computer system 1500, while inother embodiments multiple such systems, or multiple nodes making upcomputer system 1500, may be configured to host different portions orinstances of embodiments. For example, in one embodiment some elementsmay be implemented via one or more nodes of computer system 1500 thatare distinct from those nodes implementing other elements.

In various embodiments, computer system 1500 may be a uniprocessorsystem including one processor 1510, or a multiprocessor systemincluding several processors 1510 (e.g., two, four, eight, or anothersuitable number). Processors 1510 may be any suitable processor capableof executing instructions. For example, in various embodimentsprocessors 1510 may be general-purpose or embedded processorsimplementing any of a variety of instruction set architectures (ISAs),such as the x86, PowerPC, SPARC, or MIPS ISAs, or any other suitableISA. In multiprocessor systems, each of processors 1510 may commonly,but not necessarily, implement the same ISA.

System memory 1520 may be configured to store camera control programinstructions 1522 and/or camera control data accessible by processor1510. In various embodiments, system memory 1520 may be implementedusing any suitable memory technology, such as static random accessmemory (SRAM), synchronous dynamic RAM (SDRAM), nonvolatile/Flash-typememory, or any other type of memory. In the illustrated embodiment,program instructions 1522 may be configured to implement a lens controlapplication 1524 incorporating any of the functionality described above.Additionally, existing camera control data 1532 of memory 1520 mayinclude any of the information or data structures described above. Insome embodiments, program instructions and/or data may be received, sentor stored upon different types of computer-accessible media or onsimilar media separate from system memory 1520 or computer system 1500.While computer system 1500 is described as implementing thefunctionality of functional blocks of previous Figures, any of thefunctionality described herein may be implemented via such a computersystem.

In one embodiment, I/O interface 1530 may be configured to coordinateI/O traffic between processor 1510, system memory 1520, and anyperipheral devices in the device, including network interface 1540 orother peripheral interfaces, such as input/output devices 1550. In someembodiments, I/O interface 1530 may perform any necessary protocol,timing or other data transformations to convert data signals from onecomponent (e.g., system memory 1520) into a format suitable for use byanother component (e.g., processor 1510). In some embodiments, I/Ointerface 1530 may include support for devices attached through varioustypes of peripheral buses, such as a variant of the Peripheral ComponentInterconnect (PCI) bus standard or the Universal Serial Bus (USB)standard, for example. In some embodiments, the function of I/Ointerface 1530 may be split into two or more separate components, suchas a north bridge and a south bridge, for example. Also, in someembodiments some or all of the functionality of I/O interface 1530, suchas an interface to system memory 1520, may be incorporated directly intoprocessor 1510.

Network interface 1540 may be configured to allow data to be exchangedbetween computer system 1500 and other devices attached to a network1585 (e.g., carrier or agent devices) or between nodes of computersystem 1500. Network 1585 may in various embodiments include one or morenetworks including but not limited to Local Area Networks (LANs) (e.g.,an Ethernet or corporate network), Wide Area Networks (WANs) (e.g., theInternet), wireless data networks, some other electronic data network,or some combination thereof. In various embodiments, network interface1540 may support communication via wired or wireless general datanetworks, such as any suitable type of Ethernet network, for example;via telecommunications/telephony networks such as analog voice networksor digital fiber communications networks; via storage area networks suchas Fibre Channel SANs, or via any other suitable type of network and/orprotocol.

Input/output devices 1550 may, in some embodiments, include one or moredisplay terminals, keyboards, keypads, touchpads, scanning devices,voice or optical recognition devices, or any other devices suitable forentering or accessing data by one or more computer systems 1500.Multiple input/output devices 1550 may be present in computer system1500 or may be distributed on various nodes of computer system 1500. Insome embodiments, similar input/output devices may be separate fromcomputer system 1500 and may interact with one or more nodes of computersystem 1500 through a wired or wireless connection, such as over networkinterface 1540.

As shown in FIG. 15 , memory 1520 may include program instructions 1522,which may be processor-executable to implement any element or actiondescribed above. In one embodiment, the program instructions mayimplement the methods described above. In other embodiments, differentelements and data may be included. Note that data may include any dataor information described above.

Those skilled in the art will appreciate that computer system 1500 ismerely illustrative and is not intended to limit the scope ofembodiments. In particular, the computer system and devices may includeany combination of hardware or software that can perform the indicatedfunctions, including computers, network devices, Internet appliances,PDAs, wireless phones, pagers, etc. Computer system 1500 may also beconnected to other devices that are not illustrated, or instead mayoperate as a stand-alone system. In addition, the functionality providedby the illustrated components may in some embodiments be combined infewer components or distributed in additional components. Similarly, insome embodiments, the functionality of some of the illustratedcomponents may not be provided and/or other additional functionality maybe available.

Those skilled in the art will also appreciate that, while various itemsare illustrated as being stored in memory or on storage while beingused, these items or portions of them may be transferred between memoryand other storage devices for purposes of memory management and dataintegrity. Alternatively, in other embodiments some or all of thesoftware components may execute in memory on another device andcommunicate with the illustrated computer system via inter-computercommunication. Some or all of the system components or data structuresmay also be stored (e.g., as instructions or structured data) on acomputer-accessible medium or a portable article to be read by anappropriate drive, various examples of which are described above. Insome embodiments, instructions stored on a computer-accessible mediumseparate from computer system 1500 may be transmitted to computer system1500 via transmission media or signals such as electrical,electromagnetic, or digital signals, conveyed via a communication mediumsuch as a network and/or a wireless link. Various embodiments mayfurther include receiving, sending or storing instructions and/or dataimplemented in accordance with the foregoing description upon acomputer-accessible medium. Generally speaking, a computer-accessiblemedium may include a non-transitory, computer-readable storage medium ormemory medium such as magnetic or optical media, e.g., disk orDVD/CD-ROM, volatile or non-volatile media such as RAM (e.g. SDRAM, DDR,RDRAM, SRAM, etc.), ROM, etc. In some embodiments, a computer-accessiblemedium may include transmission media or signals such as electrical,electromagnetic, or digital signals, conveyed via a communication mediumsuch as network and/or a wireless link.

The methods described herein may be implemented in software, hardware,or a combination thereof, in different embodiments. In addition, theorder of the blocks of the methods may be changed, and various elementsmay be added, reordered, combined, omitted, modified, etc. Variousmodifications and changes may be made as would be obvious to a personskilled in the art having the benefit of this disclosure. The variousembodiments described herein are meant to be illustrative and notlimiting. Many variations, modifications, additions, and improvementsare possible. Accordingly, plural instances may be provided forcomponents described herein as a single instance. Boundaries betweenvarious components, operations and data stores are somewhat arbitrary,and particular operations are illustrated in the context of specificillustrative configurations. Other allocations of functionality areenvisioned and may fall within the scope of claims that follow. Finally,structures and functionality presented as discrete components in theexample configurations may be implemented as a combined structure orcomponent. These and other variations, modifications, additions, andimprovements may fall within the scope of embodiments as defined in theclaims that follow.

What is claimed is:
 1. A method, comprising: focusing a first camera onan image subject based at least in part on image content correspondingto the image subject, wherein the first camera comprises a first set ofone or more lenses that define a first optical axis and a first focallength, the focusing the first camera comprises moving, via a firstvoice coil motor (VCM) actuator of the first camera, the first set oflenses along the first optical axis to a first focus position at whichthe first camera is focused on the image subject; determining a focusposition of the first set of lenses at which the first camera is focusedon the image subject; and focusing a second camera on the image subjectbased at least in part on the focus position of the first set of lensesof the first camera and a focus relationship between the second cameraand the first camera, wherein: the second camera comprises a second setof one or more lenses that define a second optical axis and a secondfocal length, the focus relationship characterizes focus positionings ofthe second set of lenses of the second camera with respect to focuspositioning of the first set of lenses of the first camera, and thefocusing the second camera comprises moving, via a second VCM actuatorof the second camera and during a time period in which the first camerais focused on the image subject, the second set of lenses along thesecond axis to a second focus position at which the second camera isfocused on the image subject.
 2. The method of claim 1, wherein thefocus relationship comprises an offset term that is variable based atleast in part on one or more parameters corresponding to at least one ofthe first camera or the second camera.
 3. The method of claim 2, whereinthe one or more parameters comprise at least one of: a first temperatureassociated with the first set of lenses of the first camera; or a secondtemperature associated with the second set of lenses of the secondcamera.
 4. The method of claim 1, further comprising: determining thefocus relationship between the second camera and the first camera,wherein determining the focus relationship comprises: focusing the firstcamera on image subjects; obtaining, via one or more position sensors ofthe first camera, a first set of position data corresponding torespective focus positions of the first set of lenses of the firstcamera based at least in part on the focusing of the first camera on theimage subjects; focusing the second camera on the image subjects;obtaining, via one or more position sensors of the second camera, asecond set of position data corresponding to respective focus positionsof the second set of lenses of the second camera based at least in parton the focusing of the second camera on the image subjects; anddetermining the focus relationship based at least in part on the firstset of position data and the second set of position data.
 5. The methodof claim 4, further comprising: updating the focus relationship betweenthe second camera and the first camera, during a second time period thatis after a first time period in which the focus relationship isdetermined, wherein updating the focus relationship comprises: focusing,during the second time period, the first camera on at least one imagesubject; obtaining, during the second time period and via the one ormore position sensors of the first camera, a third set of position datacorresponding to one or more focus positions of the first set of lensesof the first camera based at least in part on the focusing of the firstcamera on the at least one image subject; focusing, during the secondtime period, the second camera on the at least one image subject;obtaining, during the second time period and via the one or moreposition sensors of the second camera, a fourth set of position datacorresponding to one or more focus positions of the second set of lensesof the second camera based at least in part on the focusing of thesecond camera on the at least one image subject; and determining anupdated focus relationship based at least in part on the third set ofposition data and the fourth set of position data.
 6. The method ofclaim 5, wherein the focus relationship comprises an offset term that isvariable based at least in part on one or more parameters correspondingto at least one of the first camera or the second camera, and whereinupdating the focus relationship comprises: determining a state of theone or more parameters corresponding to the second time period; andupdating the offset term of the focus relationship based at least inpart on the state of the one or more parameters.
 7. The method of claim5, wherein: the second camera is configured to focus by moving thesecond set of lenses of the second camera in search of a focus positionwithin a first focus range, focusing, during the second time period, thesecond camera on the at least one image subject comprises constrainingthe search, of one or more focus positions of the second set of lensesof the second camera at which the second camera is focused on the atleast one image subject, to a second focus range that is less than thefirst focus range; and the method further comprises: determining aconfidence level of the focus relationship; and determining the secondfocus range based at least in part on the confidence level of the focusrelationship.
 8. A system, comprising: a first camera, comprising: afirst set of one or more lenses that define a first optical axis and afirst focal length; and a first actuator configured to move the firstset of lenses along the first optical axis to enable focusing for thefirst camera; a second camera, comprising: a second set of one or morelenses that define a second optical axis and a second focal length; anda second actuator configured to move the second set of lenses along thesecond optical axis to enable focusing for the second camera; and atleast one controller configured to: focus the first camera on an imagesubject based at least in part on image content corresponding to theimage subject; determine a focus position of the first set of lenses atwhich the first camera is focused on the image subject; and focus thesecond camera on the image subject based at least in part on the focusposition of the first set of lenses of the first camera and a focusrelationship between the second camera and the first camera, wherein thefocus relationship characterizes focus positionings of the second set oflenses of the second camera with respect to focus positioning of thefirst set of lenses of the first camera.
 9. The system of claim 8,wherein the focus relationship comprises an offset term that is variablebased at least in part on one or more parameters corresponding to atleast one of the first camera or the second camera.
 10. The system ofclaim 9, wherein the one or more parameters comprise at least one of: afirst temperature associated with the first set of lenses of the firstcamera; or a second temperature associated with the second set of lensesof the second camera.
 11. The system of claim 8, wherein to determinethe focus relationship between the second camera and the first camera,the controller is configured to: focus the first camera on imagesubjects; obtain, via one or more position sensors of the first camera,a first set of position data corresponding to respective focus positionsof the first set of lenses of the first camera based at least in part onthe focusing of the first camera on the image subjects; focus the secondcamera on the image subjects; obtain, via one or more position sensorsof the second camera, a second set of position data corresponding torespective focus positions of the second set of lenses of the secondcamera based at least in part on the focusing of the second camera onthe image subjects; and determine the focus relationship based at leastin part on the first set of position data and the second set of positiondata.
 12. The system of claim 11, wherein the controller is furtherconfigured to update the focus relationship between the second cameraand the first camera, and wherein to update the focus relationship, thecontroller is configured to: focus, during a second time period that isafter a first time period in which the focus relationship is determined,the first camera on at least one image subject; obtain, during thesecond time period and via the one or more position sensors of the firstcamera, a third set of position data corresponding to one or more focuspositions of the first set of lenses of the first camera based at leastin part on the focusing of the first camera on the at least one imagesubject; focus, during the second time period, the second camera on theat least one image subject; obtain, during the second time period andvia the one or more position sensors of the second camera, a fourth setof position data corresponding to one or more focus positions of thesecond set of lenses of the second camera based at least in part on thefocusing of the second camera on the at least one image subject; anddetermine an update to the focus relationship based at least in part onthe third set of position data and the fourth set of position data. 13.The system of claim 12, wherein the focus relationship comprises anoffset term that is variable based at least in part on one or moreparameters corresponding to at least one of the first camera or thesecond camera, and wherein to update the focus relationship, thecontroller is configured to: determine a state of the one or moreparameters corresponding to the second time period; and update theoffset term of the focus relationship based at least in part on thestate of the one or more parameters.
 14. The system of claim 12, whereinthe second camera is configured to focus by moving the second set oflenses of the second camera in search of a focus position within a firstfocus range, and wherein to update the focus relationship, thecontroller is configured to: determine a confidence level of the focusrelationship; determine a second focus range, less than the first focusrange, based at least in part on the confidence level of the focusrelationship; and focus, during the second time period, the secondcamera on the at least one image subject based on constraining thesearch, of one or more focus positions of the second set of lenses ofthe second camera at which the second camera is focused on the at leastone image subject, to the second focus range.
 15. A device, comprising:a first camera, comprising: a first set of one or more lenses thatdefine a first optical axis and a first focal length; and a firstactuator configured to move the first set of lenses along the firstoptical axis to enable focusing for the first camera; a second camera,comprising: a second set of one or more lenses that define a secondoptical axis and a second focal length; and a second actuator configuredto move the second set of lenses along the second optical axis to enablefocusing for the second camera; a display; and one or more processorsconfigured to: focus the first camera on an image subject based at leastin part on image content corresponding to the image subject; determine afocus position of the first set of lenses at which the first camera isfocused on the image subject; focus the second camera on the imagesubject based at least in part on the focus position of the first set oflenses of the first camera and a focus relationship between the secondcamera and the first camera, wherein the focus relationshipcharacterizes focus positionings of the second set of lenses of thesecond camera with respect to focus positioning of the first set oflenses of the first camera; cause an image of the image subject to becaptured at least partly via one or more of the first camera or thesecond camera; and cause the display to present the image.
 16. Thedevice of claim 15, wherein the focus relationship comprises an offsetterm that is variable based at least in part on one or more parameterscorresponding to at least one of the first camera or the second camera,and wherein the one or more parameters comprise at least one of: a firsttemperature associated with the first set of lenses of the first camera;or a second temperature associated with the second set of lenses of thesecond camera.
 17. The device of claim 15, wherein to determine thefocus relationship between the second camera and the first camera, theprocessors are configured to: focus the first camera on image subjects;obtain, via one or more position sensors of the first camera, a firstset of position data corresponding to respective focus positions of thefirst set of lenses of the first camera based at least in part on thefocusing of the first camera on the image subjects; focus the secondcamera on the image subjects; obtain, via one or more position sensorsof the second camera, a second set of position data corresponding torespective focus positions of the second set of lenses of the secondcamera based at least in part on the focusing of the second camera onthe image subjects; and determine the focus relationship based at leastin part on the first set of position data and the second set of positiondata.
 18. The device of claim 17, wherein the processors are furtherconfigured to update the focus relationship between the second cameraand the first camera, and wherein to update the focus relationship, theprocessors are configured to: focus, during a second time period that isafter a first time period in which the focus relationship is determined,the first camera on at least one image subject; obtain, during thesecond time period and via the one or more position sensors of the firstcamera, a third set of position data corresponding to one or more focuspositions of the first set of lenses of the first camera based at leastin part on the focusing of the first camera on the at least one imagesubject; focus, during the second time period, the second camera on theat least one image subject; obtain, during the second time period andvia the one or more position sensors of the second camera, a fourth setof position data corresponding to one or more focus positions of thesecond set of lenses of the second camera based at least in part on thefocusing of the second camera on the at least one image subject; anddetermine an update to the focus relationship based at least in part onthe third set of position data and the fourth set of position data. 19.The device of claim 18, wherein the focus relationship comprises anoffset term that is variable based at least in part on one or moreparameters corresponding to at least one of the first camera or thesecond camera, and wherein to update the focus relationship, theprocessors are configured to: determine a state of the one or moreparameters corresponding to the second time period; and update theoffset term of the focus relationship based at least in part on thestate of the one or more parameters.
 20. The device of claim 18, whereinthe second camera is configured to focus by moving the second set oflenses of the second camera in search of a focus position within a firstfocus range, and wherein to update the focus relationship, theprocessors are configured to: determine a confidence level of the focusrelationship; determine a second focus range, less than the first focusrange, based at least in part on the confidence level of the focusrelationship; and focus, during the second time period, the secondcamera on the at least one image subject based on constraining thesearch, of one or more focus positions of the second set of lenses ofthe second camera at which the second camera is focused on the at leastone image subject, to the second focus range.